Skip to content

Packaged-app smoke test + CI - #24

Merged
ofcRS merged 2 commits into
mainfrom
feat/smoke-ci
Sep 28, 2026
Merged

ofcRS merged 2 commits into
mainfrom
feat/smoke-ci

Conversation

@ofcRS

@ofcRS ofcRS commented Sep 28, 2026

Copy link
Copy Markdown
Owner

Both crashes outside users hit, #17 at launch and #22 in Settings, happen only on a Mac without the build folder. Nothing on the build Mac could show them. This adds a test that can.

app/smoke-test.sh [Kleoth.app]

  • Runs the app as it ships, with every .build folder the binary names moved aside and restored on any exit.
  • Uses a demo-mode copy (dev.kleoth.demo, a fictional fixture meeting in app/smoke/fixture), so nothing of the user's is read or written.
  • The new smoke demo script opens the fixture meeting's page, History's Dictations and Recordings scopes, and all six Settings pages, one still each.
  • It fails on:
    • a crash;
    • two identical screens (a page that never switched);
    • the meeting page without the fixture's TL;DR;
    • Settings → General without "Record Shortcut", the recorder label from KeyboardShortcuts' own bundle.
  • About 20 s.

Wiring

  • make-app.sh release runs the smoke test before installing. A crashing build never replaces /Applications/Kleoth.app or reaches a DMG. KLEOTH_NO_INSTALL=1 stops before the install.
  • .github/workflows/tests.yml, on every PR and push to main:
    • swift test (macos-26);
    • the release app plus smoke test (macos-26), with frames uploaded as smoke-frames;
    • the marketing gate's bun tests.

Demo mode, two new gates. An audit of what Settings does when it opens found two leaks, now closed:

  • It no longer fetches the OpenRouter catalog, which wrote a cache into the real ~/.config/kleoth.
  • The personal dictionary now lives in the demo folder instead of reading the real one.

Verified

Design: docs/plans/2026-09-24-demo-mode.md §7. Limits: only the screens the script opens. The menu bar, pill, onboarding, recording and permissions are not covered.

The two crashes outside users hit (#17 at launch, #22 in Settings) only
happen on a Mac without the build folder, so nothing here could see them.
app/smoke-test.sh now runs the app as it ships with every build folder the
binary names moved aside, through a demo-mode copy (dev.kleoth.demo, a
fixture meeting, nothing of the user's read or written).

- DemoDirector `smoke`: the fixture meeting's page, History's other two
  scopes, and all six Settings pages, one still each. Fails on a crash, on
  two identical screens, without the fixture's TL;DR on the meeting page, or
  without "Record Shortcut" (KeyboardShortcuts' own bundle) on General.
- make-app.sh release runs it before installing, so a crashing build never
  replaces /Applications/Kleoth.app or reaches a DMG. KLEOTH_NO_INSTALL=1
  stops before the install (CI, branch builds).
- Demo mode now also skips the OpenRouter catalog fetch (it wrote a cache
  into ~/.config/kleoth) and keeps the personal dictionary in its folder.
- .github/workflows/tests.yml: swift test, the release app + smoke test
  (macos-26), and the marketing gate's bun tests, on every PR and push to
  main.

Red-tested: with upstream KeyboardShortcuts' Bundle.module back, the smoke
test fails at Settings → General with #22's error; with
KleothAssets.resources = Bundle.module, at History with #17's.
The first CI run failed one: a 0.05 s deadline measured 1.25 s against a
1.0 s bound. Each test's slow side is now 30 s (or its budget and retry
delay) with a bound of 4-10 s, so it still fails if the code waits for the
operation, but not on a busy machine.
@ofcRS
ofcRS merged commit 28beb9b into main Sep 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant