Skip to content

Select supported agent clients and configuration scope - #20932

Merged
Ella Hathaway (ellahathaway) merged 19 commits into
mainfrom
ellahathaway-agent-client-selection
Oct 15, 2026
Merged

Ella Hathaway (ellahathaway) merged 19 commits into
mainfrom
ellahathaway-agent-client-selection

Conversation

@ellahathaway

@ellahathaway Ella Hathaway (ellahathaway) commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

Users should choose their agents, not skill folders. agent init, new, and init now share client selection and project/global scope.

  • Use stable CLI IDs copilot, claude, and opencode, with localized GitHub Copilot, Claude Code, and OpenCode labels.
  • Prompt agents → assets → scope. Preselect detected agents; use Copilot only when none are detected.
  • Write only to selected clients and scope, deduplicating shared destinations. Selecting no agents/assets performs no agent writes. MCP remains a standalone opt-in.
  • Keep scanners read-only, client metadata focused on destinations, and configuration writes in shared helpers.

This PR provides the foundation for aspire-skills plugin support by letting users choose their agent clients and installation scope rather than filesystem locations. Clients can share skill directories while requiring different native plugin registration, so client selection determines which installation behavior is needed, while detection supplies sensible defaults. This keeps the user-facing flow stable as native plugin support is added, while preserving copied-skill installation in this intermediate layer.

This layer retains copied skills and the existing bundle. Native plugins #20946 and bundle retirement #20947 remain separate follow-ups.

aspire agent init --agents claude --scope project --skills aspire --mcp=false

Related to #20247

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 20932

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 20932"

Retain main's dead Playwright scanner cleanup and resolve the overlapping Copilot detection test using editor evidence without a fabricated CLI probe.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

OpenCode config resolution and Playwright mirroring/cleanup can produce incorrect or failed installations.

3 open findings
What changed in this PR

Replaces skill-folder selection with supported agent-client and project/user scope selection while retaining existing skill acquisition and MCP behavior.

Changes:

  • Adds Copilot, Claude, and OpenCode client selection and scoped destinations.
  • Updates chained CLI commands, MCP configuration, telemetry hooks, and Playwright staging.
  • Expands unit/E2E coverage, documentation, help text, and localization resources.
File Description
src/​Aspire.Cli/​Agents/​AgentClient.cs Defines clients and scoped paths.
src/​Aspire.Cli/​Agents/​AgentEnvironmentApplicator.cs Adds applicator target paths.
src/​Aspire.Cli/​Agents/​ClaudeCode/​ClaudeCodeAgentEnvironmentScanner.cs Supports explicit config paths.
src/​Aspire.Cli/​Agents/​Copilot/​CopilotAgentEnvironmentScanner.cs Separates VS Code detection.
src/​Aspire.Cli/​Agents/​DeprecatedMcpCommandScanner.cs Restricts repairs by target path.
src/​Aspire.Cli/​Agents/​Hooks/​TelemetryHookConfigurator.cs Honors Claude config overrides.
src/​Aspire.Cli/​Agents/​OpenCode/​OpenCodeAgentEnvironmentScanner.cs Supports explicit config paths.
src/​Aspire.Cli/​Agents/​Playwright/​PlaywrightCliInstaller.cs Stages and mirrors generated skills.
src/​Aspire.Cli/​Agents/​SkillLocation.cs Removes legacy location model.
src/​Aspire.Cli/​Commands/​AgentInitCommand.cs Implements client/scope selection.
src/​Aspire.Cli/​Commands/​InitCommand.cs Forwards agent options.
src/​Aspire.Cli/​Commands/​McpInitCommand.cs Reuses agent-init options.
src/​Aspire.Cli/​Commands/​NewCommand.cs Forwards agent options.
src/​Aspire.Cli/​Projects/​AppHostCandidateFinder.cs Updates model reference.
src/​Aspire.Cli/​README.md Documents client selection.
src/​Aspire.Cli/​Resources/​AgentCommandStrings.Designer.cs Updates generated resource members.
src/​Aspire.Cli/​Resources/​AgentCommandStrings.resx Adds client/scope strings.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.cs.xlf Updates Czech localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.de.xlf Updates German localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.es.xlf Updates Spanish localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.fr.xlf Updates French localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.it.xlf Updates Italian localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.ja.xlf Updates Japanese localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.ko.xlf Updates Korean localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.pl.xlf Updates Polish localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.pt-BR.xlf Updates Portuguese localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.ru.xlf Updates Russian localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.tr.xlf Updates Turkish localization catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.zh-Hans.xlf Updates Simplified Chinese catalog.
src/​Aspire.Cli/​Resources/​xlf/​AgentCommandStrings.zh-Hant.xlf Updates Traditional Chinese catalog.
tests/​Aspire.Cli.EndToEnd.Tests/​AgentCommandTests.cs Updates interactive agent scenarios.
tests/​Aspire.Cli.EndToEnd.Tests/​NewWithAgentInitTests.cs Updates chained-new scenarios.
tests/​Aspire.Cli.EndToEnd.Tests/​PlaywrightCliInstallTests.cs Uses client/scope options.
tests/​Aspire.Cli.Tests/​Agents/​AgentClientTests.cs Tests scoped path resolution.
tests/​Aspire.Cli.Tests/​Agents/​CommonAgentApplicatorsTests.cs Tests supported client catalog.
tests/​Aspire.Cli.Tests/​Agents/​CopilotAgentEnvironmentScannerTests.cs Tests VS Code detection.
tests/​Aspire.Cli.Tests/​Agents/​PlaywrightCliInstallerTests.cs Tests staged generation.
tests/​Aspire.Cli.Tests/​Commands/​AgentInitClientSelectionTests.cs Tests selection and scope behavior.
tests/​Aspire.Cli.Tests/​Commands/​AgentInitCommandTests.cs Updates agent-init coverage.
tests/​Aspire.Cli.Tests/​Commands/​InitCommandTests.cs Tests scoped chained init.
tests/​Aspire.Cli.Tests/​Commands/​NewCommandTests.cs Tests scoped chained new.
tests/​Aspire.Cli.Tests/​Snapshots/​AgentInitClientSelectionTests.AgentInit_DuplicateClientsAndSharedDestinations_AreReportedOnce.verified.txt Verifies deduplicated summary.
tests/​Aspire.Cli.Tests/​Snapshots/​AgentInitClientSelectionTests.AgentInit_ExplicitMcp_UsesSelectedScopeAndSharedProjectFile_scope=project.verified.json Verifies project MCP output.
tests/​Aspire.Cli.Tests/​Snapshots/​AgentInitClientSelectionTests.AgentInit_ExplicitMcp_UsesSelectedScopeAndSharedProjectFile_scope=user.verified.json Verifies user MCP output.
tests/​Aspire.Cli.Tests/​Snapshots/​InitCommandTests.InitCommand_Help_ShowsFileBasedOption.verified.txt Updates help snapshot.
tests/​Aspire.Cli.Tests/​Snapshots/​PlaywrightCliInstallerTests.InstallSelectedAsync_GeneratesOnceAndPreservesUnselectedScope_scope=project.verified.txt Verifies project mirroring.
tests/​Aspire.Cli.Tests/​Snapshots/​PlaywrightCliInstallerTests.InstallSelectedAsync_GeneratesOnceAndPreservesUnselectedScope_scope=user.verified.txt Verifies user mirroring.
tests/​Aspire.Cli.Tests/​TestServices/​TestAgentEnvironmentDetector.cs Adds detected-client test setup.
tests/​Aspire.Cli.Tests/​TestServices/​TestTelemetryHookConfigurator.cs Adds reusable hook test double.
tests/​Shared/​Hex1bAutomatorTestHelpers.cs Updates chained-flow documentation.
Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs
Comment thread src/Aspire.Cli/Agents/AgentClient.cs Outdated
Comment thread src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs Outdated
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

User-scoped deprecated MCP entries are authorized but never discovered or repaired.

4 open findings
Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced

Comment thread src/Aspire.Cli/Agents/AgentClient.cs Outdated
Use --agents and project/global scope, default to detected and universal-skill clients, and leave agent-specific legacy skill directories untouched.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
Use ID-free localized client instances, detected-agent defaults, direct bindings, and flags-based detection. Keep copied skill writes scoped to selected agents and retain standalone MCP consent.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Selected Playwright copies can fail silently, global deprecated MCP entries remain unrepaired, and the installation specification is outdated.

5 open findings
Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced

Comment thread src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs
@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
@github-actions

This comment has been minimized.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Project-directory detection can escape the selected repository boundary and use unrelated ancestor agent directories.

5 open findings
1 resolved since last review
Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced

Comment thread src/Aspire.Cli/Agents/AgentEnvironmentScanContext.cs Outdated
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

One or more issues must be addressed before approval.

0 open findings

1 resolved since last review
Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@ellahathaway

Copy link
Copy Markdown
Contributor Author

PR Testing Report

PR Information

  • PR Number: #20932
  • Title: Select supported agent clients and configuration scope
  • Head Commit: 2d4d928cba51c4aa928e7a16e87fda8951a96551
  • Tree: 157e66d7f46fee8095de928abc8249635aa97ace
  • Branch: ellahathaway-agent-client-selection
  • Base: main
  • Tested At: 2026-10-14; final result verification at 18:46:53 UTC
  • Environment: Windows ARM64, isolated temporary installation and projects

Artifact Version Verification

  • Expected Commit: 2d4d928cba51c4aa928e7a16e87fda8951a96551
  • Installed Version: 17.0.0-pr.20932.g2d4d928c
  • Artifact Run: 38605370530, completed successfully at the same full commit SHA
  • Status: ✅ Verified using the absolute downloaded executable. Local HEAD, remote branch, and live PR HEAD also matched.

This is the copied-skill client-selection layer. Native plugin installation and registration are outside this validation.

Evidence note: Log and screenshot filenames below refer to retained session-local artifacts. These files are not attached to this comment.

Changes Analyzed

Files Changed

Relevant paths Changes tested
src\Aspire.Cli\Agents\AgentClient.cs, detector/scanner contracts, and client path helpers Supported client IDs, grouped discovery, explicit undetected clients, shared destinations, and scope mapping
src\Aspire.Cli\Commands\AgentInitCommand.cs Selection, defaults, MCP consent, opt-outs, and installation scope
NewCommand.cs, InitCommand.cs, and McpInitCommand.cs Chained setup and legacy delegation
McpConfigFileHelper.cs and OpenCode\OpenCodePaths.cs JSON/JSONC precedence, pin preservation, validation, and owning-file migration
Hook configurator and Playwright installer Selected-client/scope routing while preserving existing policies
CLI resources, unit tests/snapshots, E2E consumers, and installation documentation Help, summaries, consumers, and regression coverage

The complete changed-path inventory is retained in pr-metadata.json.

Change Categories

  • CLI changes
  • Test changes
  • Hosting integration changes
  • Dashboard implementation changes
  • Template implementation changes
  • Client/component library changes
  • VS Code extension changes
  • CI infrastructure changes

Test Scenarios Executed

Scenario 1: Source build and existing focused regressions

Objective: Verify the changed CLI compiles and preserves existing agent, new/init, interaction, installer, and hook behavior.

Coverage Type: Build/unit regression

Status: ✅ Passed — 955 existing regressions.

Steps:

  1. Built Aspire.Cli.Tests and its CLI dependencies with the temporary matrix import.
  2. Ran the focused test classes through the system ARM64 .NET host.
  3. Excluded Windows-incompatible Bash tests, quarantined tests, and outerloop tests.

Evidence: source-build.log, source-tests.log, and run-source-matrix.ps1.

Observations: The build completed with zero warnings/errors. The combined source run included these 955 regressions and the 831 matrix cases below: 1,786 passed, zero failed, zero skipped, in 2m 24s 589ms. Filter-excluded tests are outside that total.


Scenario 2: Non-interactive client/scope/asset matrix

Objective: Verify every declared combination of client selection, detection-derived defaults, scope, skill selection, and MCP consent.

Coverage Type: Happy path / Negative / Boundary

Status: ✅ Passed — 765/765 cases.

Steps:

  1. Exercised nine explicit selections: all seven nonempty client subsets, all, and none.
  2. Exercised omitted client selection against all eight detection profiles, including no detected clients.
  3. Combined these with three scope modes, five skill modes, and three MCP modes: (9 + 8) × 3 × 5 × 3 = 765.
  4. Compared complete agent-file inventories, meaningful contents, preserved sentinels, MCP/hooks, acquisition counts, and exit status.

Evidence: matrix-results.csv, matrix-summary.json, and matrix-receipts\N0001.json through N0765.json.

Observations: Explicit supported clients worked without detection. Omitted selection used detected clients or the Copilot fallback. Selected clients and scope determined destinations; shared locations were coalesced, and opposite-scope/unselected-exclusive files were preserved.

Expected Unhappy-Path Outcome: none and no-assets selections leave agent configuration unchanged. Across both source matrices, 149 rows verified no agent writes.

Global shared skills were tested with a fake home. External acquisition used controlled existing test services; the 144 recorded Playwright generation calls were fake dependency calls, not real npm installations.


Scenario 3: Interactive controller matrix

Objective: Verify prompt defaults, offered choices, clearing selections, and conditional scope prompts.

Coverage Type: Happy path / Negative / Boundary

Status: ✅ Passed — 66/66 cases.

Steps:

  1. Exercised every nonempty client subset with skills-only, MCP-only, both assets, and neither asset, using both scopes where applicable.
  2. Cleared all clients and verified subsequent prompts were skipped.
  3. Accepted default choices for all eight detection profiles and checked the resulting destinations.

Evidence: matrix-results.csv, matrix-receipts\I001.json through I066.json, and source-tests.log.

Observations: Assertions verified offered choices, preselection, prompt sequence, defaults, and resulting files. These are 66 controller-fixture cases, not 66 real-terminal sessions. No tracked test/project files were changed.

Expected Unhappy-Path Outcome: Clearing clients skips later asset/scope prompts; clearing all assets skips scope and agent writes.


Scenario 4: Exact-head native CLI configuration and preservation

Objective: Verify installed PR behavior in fresh projects, including scope routing, OpenCode precedence, pins, invalid configuration, and command consumers.

Coverage Type: Happy path / Unhappy path / Boundary

Status: ✅ Passed — 37/37 scenarios in the final fresh-project rerun.

Steps:

  1. Installed and version-verified the Windows ARM64 dogfood artifact.
  2. Created fresh projects using the downloaded PR hive.
  3. Executed the scenario groups below and checked exits, inventories, preservation, and idempotence.
Scenario group Cases Result
Help and standalone MCP boundary 1 Agent options exposed; new/init do not expose MCP opt-in; hidden legacy help intentionally empty
none overrides enabled assets/Global scope 1 No agent writes
Every nonempty Project client subset 7 Correct copied-skill/MCP destinations, shared roots, and idempotence
Omitted scope/skill/MCP defaults 1 Project, bundled defaults, no MCP
Every nonempty Global MCP client subset 7 Correct supported override paths
Claude private Global skills and MCP 1 Skills and MCP written under the private override
OpenCode dual JSON/JSONC configurations 12 Pins, inherited deprecated commands, invalid containers, and precedence preserved in both scopes
Explicit OpenCode file authority 1 Adjacent sibling does not override explicit-file behavior
No assets with legacy configuration 1 No migration or writes
Invalid selections/options 1 Multiple invalid inputs fail without agent files
Legacy MCP delegation 2 Default/custom configuration behavior preserved
new chaining 1 Copied skills configured without MCP
init chaining 1 Copied skills configured while existing MCP is preserved

Evidence: native-final-results.json, native-final-scenarios.log, per-command logs, and native-final-inventory-*.json.

Observations: Inherited OpenCode pins were preserved. Effective deprecated commands were repaired in their owning JSON document rather than shadowed in JSONC. Explicit-file overrides remained authoritative.

Expected Unhappy-Path Outcome: Invalid inputs/configuration return nonzero exits and diagnostics without overwriting rejected files. Invalid inherited OpenCode containers leave both documents unchanged.

Native Global MCP used supported overrides. Native Global copied skills used Claude's private override; universal Global .agents\skills coverage remained in fake-home source fixtures.


Scenario 5: Real-terminal client/default/scope interaction

Objective: Verify representative visible prompts using the actual native executable, independently of controller fixtures.

Coverage Type: User-visible interactive / Negative / Boundary

Status: ✅ Passed — 5/5 cases.

Steps:

  1. Created fresh projects and launched the CLI in an owned terminal panel.
  2. Exercised the decisions below with actual terminal input.
  3. Checked exits, resulting files, preserved settings, and skipped prompts.
Case Observed result
Clear all clients Clearing preselected Copilot skipped remaining prompts and produced no agent files
Accept bundled defaults Seven default skills at each unique Project root; CLI-only tools unchecked; MCP No and Project scope defaults
MCP Yes with Project Claude skills and shared Project MCP written; Global override did not redirect Project writes
Select Global from scope menu Down selected Global; installation used only the private Claude override and preserved the model/custom hook
Clear all assets --skills none plus default MCP No skipped scope and agent writes

Evidence: interactive-results.json, interactive-transcript.txt, interactive-screen-excerpts.txt, interactive-file-inventory.json, and interactive-configuration-documents.json.

Observations: These five executions are additional to the 66 controller cases. The MCP-Yes run accepted Project; a fresh fixture independently verified Global using Down and Enter.

Expected Unhappy-Path Outcome: Clearing clients/assets succeeds without agent files or inappropriate later prompts.


Scenario 6: E2E consumer compilation

Objective: Verify the changed CLI E2E consumers still compile.

Coverage Type: Build validation

Status: ✅ Passed — build only; Docker execution not performed.

Steps:

  1. Ran:
dotnet build tests\Aspire.Cli.EndToEnd.Tests\Aspire.Cli.EndToEnd.Tests.csproj --no-restore /p:SkipNativeBuild=true --output artifacts\client-scope-matrix-e2e -v:quiet
  1. Inspected the build result.

Evidence: e2e-consumer-build.log.

Observations: Zero warnings/errors; completed in 3.54 seconds. Docker was unavailable, so this is compilation evidence, not a passing Docker E2E run.


Scenario 7: AppHost lifecycle and dashboard smoke

Objective: Verify a fresh exact-head application starts, can be inspected, displays its dashboard, and stops cleanly.

Coverage Type: End-to-end smoke / Empty-state boundary

Status: ✅ Passed.

Steps:

  1. Created a fresh empty C# AppHost from the downloaded PR hive.
  2. Started it using an explicit AppHost path and --isolated.
  3. Ran describe, loaded the dashboard, and captured screenshot/accessibility evidence.
  4. Closed only the owned browser tab, stopped the exact AppHost, and parsed the post-stop registry.

Evidence: smoke-start.log, smoke-describe.log, smoke-stop.log, smoke-stop-receipt.json, dashboard.png, and dashboard-snapshot.yml.

Observations: describe returned {"resources":[]}. The dashboard displayed zero resources, as expected for an empty AppHost. The post-stop isolated registry contained zero entries. The stopped dashboard's login token was redacted from retained evidence.

Expected Boundary Outcome: An empty AppHost renders an empty resource list rather than an error.

Screenshots and Recordings

  • Dashboard screenshot: dashboard.png.
  • CLI interaction evidence: Terminal transcript, rendered-screen excerpts, configuration documents, and assertions.
  • TODO: CLI prompt video/recording remains uncaptured; no asciinema/Hex1b recording was produced.

Summary

Scenario Status Notes
Source build and existing regressions ✅ Passed Zero build warnings/errors; 955 regressions
Non-interactive matrix ✅ Passed 765/765
Interactive controller matrix ✅ Passed 66/66
Native artifact ✅ Passed 37/37 final fresh-project scenarios
Real-terminal interaction ✅ Passed 5/5
E2E consumer build ✅ Passed Docker execution not run
AppHost/dashboard smoke ✅ Passed Startup, describe, empty dashboard, stop

Combined source total: 1,786/1,786 passed, comprising 955 existing regressions + 765 non-interactive cases + 66 interactive-controller cases.

Overall Result

✅ PR VERIFIED for the tested Windows ARM64 scenarios at the exact commit above.

No product defect was found in the final runs, and no production changes were needed. Initial harness compilation/temp-directory issues and four incorrect native-harness expectations were corrected in the external harness; all affected native cases were rerun in fresh projects.

Recommendations and Coverage Limits

Linux/macOS, Windows-incompatible Bash_* tests, quarantined tests, outerloop tests, and Docker E2E execution remain unvalidated by this run. E2E compilation does not replace Docker execution.

The 831-row matrix is exhaustive for its declared axes, not every filesystem failure, client version, supplemental boundary variation, or command-consumer cross-product. Source acquisition is controlled through existing fakes. Native plugin-layer behavior and a formal security review are outside this result.

PR Comment Status

Posted at the user's request. No PR title/body changes were made.

Cleanup / Inspection Status

Completed. The owned AppHost stopped, browser tab closed, and terminal exited. Temporary installations/projects and uniquely owned build outputs were removed. Evidence and exact validation commands remain in the saved report under pr20932-client-scenarios\execution.

The worktree was clean at final verification. No commit, push, branch/base/stack change, or parent/native-layer edit was performed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Valid VS Code JSONC configuration causes the Copilot MCP repair path to fail initialization.

0 open findings

Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file
Previously missed (1)

In code that hasn't changed since last review

Medium severity Use JSONC parsing for commented or trailing-comma MCP configs

src/​Aspire.Cli/​Agents/​AgentClient.cs:40

The new MCP write path reads .vscode/mcp.json with Copilot's default strict JSON options before checking whether a repair is needed. VS Code allows comments and trailing commas in this file, so valid editor configuration makes agent init --agents copilot --scope project --skills none --mcp report malformed JSON and exit unsuccessfully, even without a deprecated entry. Use ConfigurationHelper.ParseOptions for these reads and add a regression covering a commented configuration and a trailing comma.

🧠 Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-client configuration precedence and persistent global configuration and hook writes warrant final human validation.

0 open findings

Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Comment thread src/Aspire.Cli/Agents/McpConfigFileHelper.cs Outdated
@JamesNK

Copy link
Copy Markdown
Member

This changes what the user sees in the CLI, right? Can you add a record or screenshots of what it looks now.

@ellahathaway

Ella Hathaway (ellahathaway) commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

This changes what the user sees in the CLI, right? Can you add a record or screenshots of what it looks now.

James Newton-King (@JamesNK) It does. Below are some screenshots and video demos of the new UX

Video

agent-init-demp.mp4

agent init --help Output

Screenshot 2026-10-15 104102

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0f73663-c37c-4c5a-b83d-89f0b8d13c5f
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector

3 / 99 PR test projects · 4 PR jobs, from 98 changed files.

Selected PR test projects (3 / 99)

Aspire.Cli.EndToEnd.Tests, Aspire.Cli.Tests, Infrastructure.Tests

Selected PR jobs (4)

cli-starter-validation, extension-e2e, polyglot, typescript-api-compat


How these were chosen — grouped by what changed

📦 affected project Aspire.Cli
→ 1 test: Aspire.Cli.EndToEnd.Tests

🔧 src/Aspire.Cli/Resources/AgentCommandStrings.resx (changed source)
→ 1 directly: Infrastructure.Tests

🧪 tests/Aspire.Cli.EndToEnd.Tests/AgentCommandTests.cs (changed test)
→ 1 directly: Aspire.Cli.EndToEnd.Tests

🧪 tests/Aspire.Cli.EndToEnd.Tests/NewWithAgentInitTests.cs (changed test)
→ 1 directly: Aspire.Cli.EndToEnd.Tests

🧪 tests/Aspire.Cli.EndToEnd.Tests/PlaywrightCliInstallTests.cs (changed test)
→ 1 directly: Aspire.Cli.EndToEnd.Tests

🧪 tests/Aspire.Cli.Tests/Agents/AgentEnvironmentApplicatorTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/AgentEnvironmentDetectorTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/ClaudeCodeAgentEnvironmentScannerTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/CommonAgentApplicatorsTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/CopilotAgentEnvironmentScannerTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/OpenCodeAgentEnvironmentScannerTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/PlaywrightCliInstallerTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/TelemetryHookConfiguratorTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Agents/VsCodeAgentEnvironmentScannerTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Commands/AgentInitClientSelectionTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Commands/AgentInitCommandTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Commands/InitCommandTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Commands/NewCommandTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Interaction/ConsoleInteractionServiceTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_AliasedMcpDestinations_ConfigureEachSelectedServerContainer_pinned=False.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_AliasedMcpDestinations_ConfigureEachSelectedServerContainer_pinned=True.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DeprecatedMcpCommand_RepairsSelectedConfigurationWithoutReplacingOtherSettings_clientId=claude_scopeName=global.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DeprecatedMcpCommand_RepairsSelectedConfigurationWithoutReplacingOtherSettings_clientId=claude_scopeName=project.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DeprecatedMcpCommand_RepairsSelectedConfigurationWithoutReplacingOtherSettings_clientId=copilot_scopeName=global.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DeprecatedMcpCommand_RepairsSelectedConfigurationWithoutReplacingOtherSettings_clientId=copilot_scopeName=project.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DeprecatedMcpCommand_RepairsSelectedConfigurationWithoutReplacingOtherSettings_clientId=opencode_scopeName=global.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DeprecatedMcpCommand_RepairsSelectedConfigurationWithoutReplacingOtherSettings_clientId=opencode_scopeName=project.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_DuplicateClientsAndSharedDestinations_AreReportedOnce.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_ExplicitMcp_UsesSelectedScopeAndSharedProjectFile_scope=global.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_ExplicitMcp_UsesSelectedScopeAndSharedProjectFile_scope=project.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=False_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=True_configureMcp=False.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=True_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=False_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=True_configureMcp=False.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=True_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeConfiguration_UsesExistingFormatAndPrecedence_scopeName=global_jsonExists=False_jsoncExists=False.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeConfiguration_UsesExistingFormatAndPrecedence_scopeName=global_jsonExists=True_jsoncExists=False.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeConfiguration_UsesExistingFormatAndPrecedence_scopeName=global_jsonExists=True_jsoncExists=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeConfiguration_UsesExistingFormatAndPrecedence_scopeName=project_jsonExists=False_jsoncExists=False.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeConfiguration_UsesExistingFormatAndPrecedence_scopeName=project_jsonExists=True_jsoncExists=False.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeConfiguration_UsesExistingFormatAndPrecedence_scopeName=project_jsonExists=True_jsoncExists=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeJsonc_PreservesExistingSettingsWhenAddingServer_scopeName=global.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeJsonc_PreservesExistingSettingsWhenAddingServer_scopeName=project.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeSiblingJson_RepairsInheritedDeprecatedCommand_scopeName=global_partialOverride=False.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeSiblingJson_RepairsInheritedDeprecatedCommand_scopeName=global_partialOverride=True.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeSiblingJson_RepairsInheritedDeprecatedCommand_scopeName=project_partialOverride=False.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_OpenCodeSiblingJson_RepairsInheritedDeprecatedCommand_scopeName=project_partialOverride=True.verified.json (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/InitCommandTests.InitCommand_Help_ShowsFileBasedOption.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/PlaywrightCliInstallerTests.InstallSelectedAsync_GeneratesOnceAndPreservesUnselectedScope_scope=global.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/PlaywrightCliInstallerTests.InstallSelectedAsync_GeneratesOnceAndPreservesUnselectedScope_scope=project.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/TestServices/FakePlaywrightServices.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/TestServices/TestAgentEnvironmentDetector.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/TestServices/TestAgentEnvironmentScanner.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/TestServices/TestExtensionInteractionService.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/TestServices/TestInteractionService.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/TestServices/TestTelemetryHookConfigurator.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

Job reasons

Job Triggered by
cli-starter-validation affected project Aspire.Cli
extension-e2e • src/Aspire.Cli/Agents/AgentClient.cs, src/Aspire.Cli/Agents/AgentClientKind.cs, src/Aspire.Cli/Agents/AgentEnvironment.cs, src/Aspire.Cli/Agents/AgentEnvironmentApplicator.cs, src/Aspire.Cli/Agents/AgentEnvironmentDetector.cs, src/Aspire.Cli/Agents/AgentEnvironmentScanContext.cs, src/Aspire.Cli/Agents/ClaudeCode/ClaudeCodeAgentEnvironmentScanner.cs, src/Aspire.Cli/Agents/ClaudeCode/ClaudeCodePaths.cs, src/Aspire.Cli/Agents/Copilot/CopilotAgentEnvironmentScanner.cs, src/Aspire.Cli/Agents/DeprecatedMcpCommandScanner.cs, src/Aspire.Cli/Agents/Hooks/ITelemetryHookConfigurator.cs, src/Aspire.Cli/Agents/Hooks/TelemetryHookConfigurator.cs, src/Aspire.Cli/Agents/IAgentEnvironmentDetector.cs, src/Aspire.Cli/Agents/IAgentEnvironmentScanner.cs, src/Aspire.Cli/Agents/McpConfigFileHelper.cs, src/Aspire.Cli/Agents/McpInitPromptGroup.cs, src/Aspire.Cli/Agents/OpenCode/OpenCodeAgentEnvironmentScanner.cs, src/Aspire.Cli/Agents/OpenCode/OpenCodePaths.cs, src/Aspire.Cli/Agents/Playwright/PlaywrightCliInstaller.cs, src/Aspire.Cli/Agents/SkillLocation.cs, src/Aspire.Cli/Agents/VsCode/VsCodeAgentEnvironmentScanner.cs, src/Aspire.Cli/Commands/AgentInitCommand.cs, src/Aspire.Cli/Commands/InitCommand.cs, src/Aspire.Cli/Commands/McpInitCommand.cs, src/Aspire.Cli/Commands/NewCommand.cs, src/Aspire.Cli/Program.cs, src/Aspire.Cli/Projects/AppHostCandidateFinder.cs, src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs, src/Aspire.Cli/Resources/AgentCommandStrings.resx, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.cs.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.de.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.es.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.fr.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.it.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ja.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ko.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.pl.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.pt-BR.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ru.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.tr.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.zh-Hans.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.zh-Hant.xlf
• affected project Aspire.Cli
polyglot affected project Aspire.Cli
typescript-api-compat affected project Aspire.Cli

Selection computed for commit 63e2efa.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-client configuration migrations and global filesystem writes require final human validation beyond the static review.

0 open findings

Files not reviewed (1)
  • src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file

🧠 Review effort: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@ellahathaway
Ella Hathaway (ellahathaway) merged commit c9a9184 into main Oct 15, 2026
568 of 580 checks passed
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 17.0 milestone Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants