Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,48 @@
build/DerivedData/Logs/Test/*.xcresult
~/Library/Logs/DiagnosticReports/Flotilla*
retention-days: 7

# Parallel to `build`, so UI tests don't lengthen a run. A fresh runner, too:
# nothing has launched the app at a smaller display size first.
ui-tests:
runs-on: xcode-27
timeout-minutes: 45

env:
SIGNING_OVERRIDE: CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO DEVELOPMENT_TEAM=

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install XcodeGen, tmux and displayplacer
run: brew install xcodegen tmux displayplacer

- name: Install Metal toolchain
run: xcodebuild -downloadComponent MetalToolchain

# The runner boots at 1024x768, narrower than the main window's
# minimum width, which leaves the toolbar and sidebars off-screen.
- name: Use a 1920x1080 display
run: |
id=$(displayplacer list | awk '/^Persistent screen id:/ {print $4; exit}')
displayplacer "id:$id res:1920x1080"
displayplacer list | grep -m1 '^Resolution:'

- name: Generate Xcode project
run: xcodegen generate

- name: Run UI tests
run: >
make test-ui SIGNING_OVERRIDE="$SIGNING_OVERRIDE"
TEST_FLAGS="-test-timeouts-enabled YES -maximum-test-execution-time-allowance 180"

- name: Upload test results
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: ui-test-results
path: |
build/DerivedData/Logs/Test/*.xcresult
~/Library/Logs/DiagnosticReports/Flotilla*
retention-days: 7
Comment on lines +68 to +108
39 changes: 14 additions & 25 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,8 @@ func testWorktreeCreationFailureDoesNotCreateSession() { ... }
> - You are explicitly asked to run or verify UI tests
> For most changes, a Debug build + unit tests is sufficient verification. When running UI tests, prefer the minimal targeted set of test classes over the full suite (`-only-testing:FlotillaUITests/<SpecificClass>`).

**Run them from a normal desktop Space, not while another app is full screen.** The test app's window then opens in a Space that isn't visible: clicks and queries still work, but any `screenshot()` fails with "Image creation failed".

**Scope: only write a UI test for something a human can't easily eyeball, or that's a stable behavioral contract.** Don't write UI tests asserting on visual/styling details — background tints, padding, corner radius, colors, frame sizes chosen for looks. A human glances at a screenshot and knows instantly if those are wrong; a test asserting on them breaks on every intentional visual change and becomes pure maintenance overhead with no signal. Reserve UI tests for things that are easy to silently regress and hard to notice by eye: keyboard-shortcut wiring, focus/dismissal behavior, data flowing correctly into fields, multi-step interaction sequences, accessibility-identifier contracts other tests depend on.

- All UI tests set `UI_TESTING=1` environment variable
Expand All @@ -418,25 +420,6 @@ DeleteSessionDialog.Cancel
DeleteSessionDialog.DeleteWithWorktree
```

### UI Vocabulary Screenshot Pipeline

`VocabularyScreenshotUITests` is a documentation generator, not a visual-regression test. Its purpose is to keep the visual appendix in `docs/ui-vocabulary.md` aligned with the app's current UI so contributors can connect canonical vocabulary such as **navigation rail**, **grid presentation**, and **diff panel** to the actual interface. It uses the deterministic `UI_TESTING=1` fixtures and deliberately makes no pixel-level styling assertions.

To refresh the screenshots, open the project in Xcode, select the **UI Vocabulary Screenshots** scheme, and choose **Product → Test**. Run this workflow through Xcode rather than `xcodebuild` in a terminal because macOS UI automation requires Accessibility permission from the process driving the test. The dedicated scheme is declared in `project.yml`, selects only `VocabularyScreenshotUITests`, disables parallel execution, and runs `Scripts/update-ui-vocabulary-screenshots.sh` as its post-test action. Running the screenshot class through another scheme captures raw files but does not publish them.

The pipeline works as follows:

1. The test launches Flotilla with deterministic fixtures, moves capture windows onto the primary display, navigates the documented surfaces, and saves both persistent XCTest attachments and raw Retina PNGs in the UI-test runner's sandbox at `~/Library/Containers/com.niclassslua.FlotillaUITests.xctrunner/Data/Documents/flotilla-vocab-shots/`.
2. Class teardown writes `manifest.txt`. It contains `COMPLETE schema=1` only when every image required by the documentation was captured successfully.
3. The scheme post-action runs `Scripts/update-ui-vocabulary-screenshots.swift`. The publisher validates the completed manifest and all required inputs, derives the documented crops, retains high-resolution output (up to 3,200 px wide for full-window images), and stages the complete 21-image set before writing to `docs/images/ui-vocabulary/`. It then rewrites the visual-reference figures in `docs/ui-vocabulary.md` between the `BEGIN/END GENERATED FIGURES` markers — one image per line, from the same `publications` list — so that section never needs hand-editing and can't drift into multi-column tables. Prose outside the markers is untouched.
4. If the test is partial, a required capture is missing, or rendering fails, publication stops and the checked-in documentation images and Markdown remain unchanged.

After changing a documented surface, capture name, caption, or visual-reference entry, update these together: `VocabularyScreenshotUITests.requiredCaptureNames` and the `publications` list in `Scripts/update-ui-vocabulary-screenshots.swift` (which carries each figure's `section`, `caption`, `alt`, and crop). The `docs/ui-vocabulary.md` figure blocks are regenerated from that list — don't edit them by hand; edit any surrounding prose directly. Run the dedicated scheme, inspect the refreshed PNGs visually, and commit the image and Markdown changes with the code. To rerun only publication from the most recent complete capture, use:

```bash
/bin/bash Scripts/update-ui-vocabulary-screenshots.sh
```

### Accessibility

- Extensive use of `.accessibilityIdentifier()` with dot-notation namespacing
Expand Down Expand Up @@ -543,12 +526,18 @@ Publishing a GitHub Release archives the app with `make archive`, wraps `Flotill
(also uploaded as the `Flotilla-DMG` artifact). Notarization is not included
yet — Gatekeeper will still warn on first open until that is added.

CI does not run `make test-ui` yet. UI automation itself works on the hosted
runner, but its display boots at 1024x768, narrower than the main window's
minimum width. Switching it to 1920x1080 (`displayplacer`) still leaves the
window 692 pt tall, and 11 of 30 UI tests fail on content that doesn't fit.
Run the targeted `make test-ui` locally before merging changes that affect
UI/accessibility.
A parallel `ui-tests` job runs `make test-ui` on its own runner (results in
the `ui-test-results` artifact). The runner boots at 1024x768, narrower than
the main window's minimum width, so the job switches the display to 1920x1080
with `displayplacer` first. XCUITest drags never reach the Home grid's
`DragGesture`, so widget reordering is tested via the context menu and the
drag itself stays a manual check.

An `.accessibilityIdentifier` on a plain container is applied to every
descendant, overriding their own identifiers — UI tests then find the wrong
element or none. Give identified containers
`.accessibilityElement(children: .contain)`, and icon+text labels
`.accessibilityElement(children: .combine)`.

Debug signs with the maintainer's own Apple Development team
(`DEVELOPMENT_TEAM: UWAHVC4JTL` in `project.yml`) so that Screen Recording
Expand Down
4 changes: 0 additions & 4 deletions Flotilla.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,6 @@
3F3691E5A9D184DE03C5614C /* WorkspaceFileDiscovery.swift in Sources */ = {isa = PBXBuildFile; fileRef = 71BFFA35FF318BF4E181C0FE /* WorkspaceFileDiscovery.swift */; };
3F8C871B8FD16CEA3EE5FC5D /* SessionGitSidebarViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3F7E6D3E8923F89B47F5C5A4 /* SessionGitSidebarViewModel.swift */; };
3FD1D7A44C78DD4C5F14C3A7 /* SessionScrollbackStore.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDF3E0DEEB5F96FA1C459ACF /* SessionScrollbackStore.swift */; };
3FF408A7A83C6BFB8FF50AC0 /* VocabularyScreenshotUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = FA3EC024D3B5C2BB547EA212 /* VocabularyScreenshotUITests.swift */; };
40D55747F5B1B96029268306 /* ProjectOverviewViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 359C3DC13E13F1E59F8BBC43 /* ProjectOverviewViewModel.swift */; };
40E4144C522F51072D9D5345 /* GridLayoutEngine.swift in Sources */ = {isa = PBXBuildFile; fileRef = 620CED7B87823AB08CD08B83 /* GridLayoutEngine.swift */; };
410AA4AE51FDBB909BE561BF /* CompanionHostStorage.swift in Sources */ = {isa = PBXBuildFile; fileRef = 081F4CCD57333F5F378F9464 /* CompanionHostStorage.swift */; };
Expand Down Expand Up @@ -1333,7 +1332,6 @@
F826B96C866E5F1F1BEDB070 /* GridDimensionsPicker.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GridDimensionsPicker.swift; sourceTree = "<group>"; };
F88CCB0EF5E481D579CB0EDB /* LedgerDesign.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LedgerDesign.swift; sourceTree = "<group>"; };
F9F0079146D1AF9E1A0A6F9F /* AppEnvironment.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppEnvironment.swift; sourceTree = "<group>"; };
FA3EC024D3B5C2BB547EA212 /* VocabularyScreenshotUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VocabularyScreenshotUITests.swift; sourceTree = "<group>"; };
FA83C6EA780D98B29510CDE6 /* folder-server.svg */ = {isa = PBXFileReference; path = "folder-server.svg"; sourceTree = "<group>"; };
FAC321D9AF789EF5234D13AB /* FleetActivityController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FleetActivityController.swift; sourceTree = "<group>"; };
FAEF1F1F164CC43EFF1FE78E /* CIStatusStore.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CIStatusStore.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -1477,7 +1475,6 @@
E4A98AEF98A101073B3ECE81 /* SettingsUITests.swift */,
8A5491FDE4DC8E390D1F339D /* StartupCheckUITests.swift */,
D9A4531D93B3A63280F51879 /* TerminalUITests.swift */,
FA3EC024D3B5C2BB547EA212 /* VocabularyScreenshotUITests.swift */,
);
path = FlotillaUITests;
sourceTree = "<group>";
Expand Down Expand Up @@ -3451,7 +3448,6 @@
65A0C5DF6D0EA6406B9CE77A /* SettingsUITests.swift in Sources */,
D65DAD38EB19053828B3A1F6 /* StartupCheckUITests.swift in Sources */,
94DB32B0489F507550BE625C /* TerminalUITests.swift in Sources */,
3FF408A7A83C6BFB8FF50AC0 /* VocabularyScreenshotUITests.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
Expand Down

This file was deleted.

11 changes: 8 additions & 3 deletions Flotilla/App/AppEnvironment.swift
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ final class AppEnvironment {
/// only the app process can.
static let uiTestFixtureProjectPath = URL(fileURLWithPath: "/tmp/flotilla-uitest-project")
static let uiTestWorktreeBasePath = URL(fileURLWithPath: "/tmp/flotilla-uitest-worktrees")
static let uiTestFixtureWorktreePath = URL(fileURLWithPath: "/tmp/flotilla-fixture-project-worktrees/fix-login-bug")

let sessionRepository: SessionRepository
let gitService: GitServiceProtocol
Expand Down Expand Up @@ -90,8 +91,7 @@ final class AppEnvironment {

private static func resetUITestWorktreeDirectories() {
let fileManager = FileManager.default
let fixtureWorktree = URL(fileURLWithPath: "/tmp/flotilla-fixture-project-worktrees/fix-login-bug")
for path in [uiTestWorktreeBasePath, URL(fileURLWithPath: "/tmp/flotilla-custom-worktrees"), fixtureWorktree] {
for path in [uiTestWorktreeBasePath, URL(fileURLWithPath: "/tmp/flotilla-custom-worktrees")] {
try? fileManager.removeItem(at: path)
try? fileManager.createDirectory(at: path, withIntermediateDirectories: true)
}
Expand Down Expand Up @@ -193,6 +193,11 @@ final class AppEnvironment {
try "# Fixture Rules\n\nBe concise.\n".write(to: path.appendingPathComponent("CLAUDE.md"), atomically: true, encoding: .utf8)
try runGit(["add", "README.md", "CLAUDE.md"], at: path)
try runGit(["-c", "user.name=Flotilla UITests", "-c", "user.email=uitests@example.com", "commit", "-m", "init"], at: path)

// A real registered worktree, not a bare directory: deleting a session
// refuses to remove a folder git doesn't know as a worktree.
try? fileManager.removeItem(at: uiTestFixtureWorktreePath)
try runGit(["worktree", "add", "-b", "flotilla/fix-login-bug", uiTestFixtureWorktreePath.path], at: path)
}

private static func runGit(_ arguments: [String], at path: URL) throws {
Expand Down Expand Up @@ -245,7 +250,7 @@ final class AppEnvironment {
workingDirectory: project.rootPath,
worktree: WorktreeInfo(
branchName: "flotilla/fix-login-bug",
worktreePath: URL(fileURLWithPath: "/tmp/flotilla-fixture-project-worktrees/fix-login-bug"),
worktreePath: uiTestFixtureWorktreePath,
baseCheckoutPath: project.rootPath
),
status: .working
Expand Down
5 changes: 2 additions & 3 deletions Flotilla/App/UITestWindowPlacer.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@
import AppKit
import SwiftUI

/// Keeps screenshot-producing UI tests on the primary display.
/// Keeps UI-test windows on the primary display, at a known size.
///
/// `XCUIScreenshot` cannot reliably capture windows hosted on a secondary
/// display. This representable is only mounted when `UI_TESTING=1`, so normal
/// This representable is only mounted when `UI_TESTING=1`, so normal
/// app launches retain macOS' standard window placement behavior.
enum UITestWindowPlacement: Equatable {
case fillPrimaryDisplay
Expand Down
4 changes: 4 additions & 0 deletions Flotilla/Features/Home/Widgets/HomeWidgetCard.swift
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ struct HomeWidgetCard<Content: View>: View {
// a click between the content's own controls lands on nothing, so
// neither dragging nor the context menu would ever start.
.contentShape(HomeWidgetCardMetrics.shape)
// `.contain` keeps this identifier on the card itself; on a plain
// container SwiftUI stamps it onto every child instead, so queries
// for the card matched its header icon.
.accessibilityElement(children: .contain)
.accessibilityIdentifier(AXID.homeWidget.rawValue + kind.rawValue)
}

Expand Down
Loading
Loading