Skip to content

ci: cover template overlays in CLI install/build e2e - #1152

Merged
AbhinRustagi merged 4 commits into
mainfrom
devin/1789034402-cli-e2e-overlays
Sep 17, 2026
Merged

AbhinRustagi merged 4 commits into
mainfrom
devin/1789034402-cli-e2e-overlays

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What

CLI End to End only ever scaffolded the default overlay, so overlay dependency sets (LangGraph, Vercel AI SDK, Vercel Eve) and their lockfile handling in create-app were never installed or built in CI. This adds overlay coverage without multiplying the existing 26-job OS × package-manager matrix by 4.

The new jobs immediately caught two real problems, fixed here:

  1. openui-cloud's vercel-eve overlay failed next build — its tsconfig was missing the allowImportingTsExtensions / exclude: ["agent"] that the self-hosted overlay has, so agent/tools/get_weather.ts (which imports ../../src/lib/tools/get-weather.ts by design, for Eve's ESM module map) tripped TS5097.
  2. The e2e jobs were not actually testing the checkout's templates: checkoutSource/fetchSourceFile always fetch thesysdev/openui@main, so a PR touching templates/** ran its CLI against main's templates.

Changes

  • Extracted the scaffold → verify-package-manager → build steps into a composite action .github/actions/cli-scaffold-e2e, parameterized by template, backend-framework, node-version, package-manager, package-manager-version. The CLI invocation now always passes --backend-framework.
  • New cli-e2e-overlays job: {openui-cloud, openui-self-hosted} × {langgraph, vercel-ai-sdk, vercel-eve} × {npm, pnpm} = 12 jobs on ubuntu / Node 24 / package-manager major 11. Overlays only change dependencies and app code, so the OS and package-manager-major spread stays on the default overlay (cli-e2e, unchanged in coverage).
  • CLI: OPENUI_SOURCE_DIR env var makes checkoutSource/fetchSourceFile read templates and templates.json from a local source checkout instead of GitHub main; the composite action sets it to github.workspace so the jobs test the commit under test. Unset (i.e. for users) behavior is unchanged.
  • templates/openui-cloud/overlays/vercel-eve/tsconfig.json: added allowImportingTsExtensions and excluded agent, matching the self-hosted overlay.
  • Workflow paths filters now include the composite action.

Note the composite action forces shell: bash for the run steps (composite steps must declare a shell); on the Windows jobs those steps previously defaulted to pwsh. The merge with main carried its pnpm setup fixes (the package_json_file pin and the runner.temp version check) into the composite action.

Test Plan

  • actionlint clean
  • pnpm --filter @openuidev/cli lint:check and build pass after merging main
  • Locally packed the CLI and ran openui create --template openui-cloud --backend-framework vercel-eve with OPENUI_SOURCE_DIR pointing at the checkout: scaffold + npm install + next build all pass with the tsconfig fix (this also exercises the new local-source path)
  • The workflow's own paths filter makes this PR run the full matrix, including the 12 new overlay jobs

Checklist

  • I linked a related issue, if applicable
  • I updated docs/README when needed (no doc surface changed)
  • I considered backwards compatibility

Link to Devin session: https://app.devin.ai/sessions/c594d3982aa447a39e419b32bdea004a
Open in Devin Desktop: https://app.devin.ai/desktop/session/c594d3982aa447a39e419b32bdea004a?variant=devin

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
openui-docs Ready Ready Preview Sep 17, 2026 2:29am UTC

Request Review

…ocal templates in CI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Resolve cli-e2e.yml conflict: keep the composite-action refactor and overlay
matrix, and port main's pnpm setup fixes (package_json_file pin and the
runner.temp version check) into the composite action.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…State

Drop the removed eve/client SessionState import in the cloud vercel-eve overlay, mirroring the self-hosted overlay.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@AbhinRustagi
AbhinRustagi merged commit f21a157 into main Sep 17, 2026
44 checks passed
@AbhinRustagi
AbhinRustagi deleted the devin/1789034402-cli-e2e-overlays branch September 17, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant