[Richie Gomez (richiemsft)]

📖 Description

This PR improves execution telemetry and correlation across a container lifecycle, and makes the development 1.1.0-alpha contract collect only four Required diagnostic events by default, without MXC consent.

• Replace MXC.Execution with MXC.ExecutionStart and MXC.ExecutionEnd, including streaming, PTY, and container lifecycle paths.
• Add mxc.session_id to the event families that remain. Reuse the persisted random lifecycle root across executor processes and preserve attribution across streaming handles, worker threads, and cleanup.
• Add the bounded mxc.capture_denials mode to Start/End and reuse the existing capability classifier.
• For development 1.1.0-alpha requests, emit only MXC.ExecutionStart, MXC.ExecutionEnd, MXC.Error, and MXC.ProcessExited, with Critical Data and Product and Service Performance metadata. The other seven event families are not emitted for this contract. Published 0.9.0-alpha and 1.0.0 requests keep their full event set and Measures/Usage metadata.
• Development requests do not need MXC consent or the request telemetry.enabled flag (the field is still accepted, including false). Collection requires a build routed through the WinExt group (or the local-only test mode) and an administrative policy that is absent, 1, or 3. Policy 0, unsupported values, wrong types, and read failures block collection. Published contracts keep request opt-in, MXC consent, and their existing policy handling.
• Use a single WinExt registration under the MXC privacy product for all events. MXC_TELEMETRY_PROVIDER_GROUP_GUID is the only routing input; a nonempty MXC_TELEMETRY_CORE_PROVIDER_GROUP_GUID now fails the Windows build.
• Restrict MXC_TELEMETRY_TEST_KEYWORDS=1 to local-only Windows debug/test-support builds. Public CI clears the routing group and validates actual keyword/privacy metadata without Microsoft routing.
• Replace docs/telemetry.md with the Required-data description and update the architecture and CI documentation and the ETW validation scripts.

Collectors must update filters and queries from MXC.Execution to the new Start/End names. Start describes a pre-dispatch attempt, not a successful process launch; missing End is not definitive evidence of a crash.

Privacy: Privacy review approved these four events as Required diagnostic data for the documented purposes. The approval does not cover additional collection. Privacy also asked how developers who integrate MXC are told that it collects Required diagnostics; that notice is not part of this PR.

Publication: No private WinExt group value is committed. The group identifier is injected through a private process-local build input. Real-group binaries, generated provider files, and VM capture artifacts are not part of this source change and must not be published as public artifacts.

🔗 References

Refs #1588. Refs #1517.

• Consumer telemetry guidance
• Telemetry architecture
• Consent design
• CI validation runbook

🔍 Validation

Local validation of the final change, with MXC_TELEMETRY_TEST_KEYWORDS=1 and a fake routing group:

Command	Result
From src: cargo fmt --all -- --check, and rustfmt --check on provider_codegen.rs	Passed.
From src: cargo test --locked -p mxc-sdk --lib --features test-support,isolation_session -- mxc_common::telemetry:: mxc_telemetry:: mxc_engine:: mxc_common::exec_stream::tests::exec_waiter_preserves_telemetry_context piped_exec_workers_preserve_telemetry_authorization_and_attribution	342 passed, 1 failed. The failure was correlation_state::requested_long_lived_record_is_refreshed..., a timing-sensitive test outside this change; it passes when run alone.
From src: policy and provider selectors	43 passed, including the new required-collection policy matrix.
From src: cargo test --locked -p wxc --bin wxc-exec --features test-support,isolation_session	159 passed.
From src: cargo clippy --locked -p mxc-sdk -p wxc --all-targets --features test-support,isolation_session -- -D warnings	Passed.
From src: cargo build --locked -p wxc --features test-support,isolation_session	Passed.
tests\scripts\test_telemetry_etw_assertions.ps1 -WxcExePath src\target\debug\wxc-exec.exe	Passed.
tests\scripts\run_telemetry_consent_smoke_test.ps1 -BinDir src\target\debug	Passed with isolated consent/policy redirects.
node scripts\check-telemetry-policy-parity.js	Passed.

Manual live validation on an internal Windows x64 VM, using a release executor built with the real WinExt group (value not recorded here):

• Sent development 1.1.0-alpha requests with telemetry.enabled=false, one successful ProcessContainer run and one unsupported-backend failure each. Published 1.0.0 requests with enabled=true were the control.
• Consent granted, withdrawn, and undetermined: development requests emitted exactly the four Required events, each with Critical Data and Performance metadata. None of the other seven events appeared.
• Published control: emitted its full event set with consent granted and nothing with consent withdrawn or undetermined.
• Policy 0: HKLM\SOFTWARE\Policies\Mxc\AllowTelemetry=0 produced no MXC events for either contract. The Telemetry RealTime Tool showed no MXC events for that window.
• The VM was restored afterward: policy key removed and consent returned to its prior state. Windows diagnostic settings were not changed.

Not established: whether UTC accepts these events from the WinExt registration. In the Telemetry RealTime Tool the development events showed Critical category and persistence but isRequired: false. An earlier run with a separate Core registration was rejected by UTC with IncorrectProductOrDataCategory; that registration is removed in this change and WinExt acceptance has not been re-checked in Diagnostic Data Viewer. Local ETW is not evidence of Microsoft ingestion.

The scheduled IsolationSession lifecycle scenario is wired into CI but was not run on the VM. Non-Windows runtime validation and SDK integration suites were not rerun.

✅ Checklist

• Signed the Contributor License Agreement
• Linked to an issue
• For features or changes requiring a spec: linked the issue with maintainer-applied feature-accepted
• For implementation: linked the required approved and merged design/API spec, or explained why no spec is required
• Feature issue remains open unless this PR completes all agreed implementation, validation, and documentation
• Updated documentation (if applicable)
• Updated Copilot instructions (if build, architecture, or conventions changed)
• If this PR changes Cargo.lock, the dependency-feed-check check passes (see pull request builds)

📋 Issue Type

• Bug fix
• Feature
• Task

Microsoft Reviewers: Open in CodeFlow

[Copilot]

🟡 Changes recommended

Invocation context is lost across backend thread boundaries, and missing session IDs incorrectly suppress authorized streaming telemetry.

2 open issues

• Propagate session context across backend worker threads · New
• Preserve authorization state when resuming handles with unknown IDs · New

What changed in this PR

This PR adds execution Start/End telemetry and lifecycle-wide session correlation across MXC executors and SDK operations.

Changes:

• Replace MXC.Execution with Start/End events and add session IDs and bounded denial-capture modes.
• Preserve attribution across streaming-handle operations and persist lifecycle correlation roots.
• Update consent disclosure to version 4, documentation, and regression coverage.

File	Description
tests/​scripts/​run_telemetry_etw_smoke_test.ps1	Validate execution pairs, attribution, and consent v4.
tests/​scripts/​run_telemetry_consent_smoke_test.ps1	Update consent expectations to v4.
tests/​scripts/​run_telemetry_consent_release_test.ps1	Update release consent checks to v4.
src/​tools/​wxc/​src/​main.rs	Initialize session attribution and emit execution Start.
src/​tools/​mxc_darwin/​src/​main.rs	Add the execution Start call.
src/​tools/​lxc/​src/​main.rs	Add the execution Start call.
src/​mxc-sdk/​tests/​wxc_e2e_tests_e2e_telemetry_etw.rs	Check new event names and attribution fields.
src/​mxc-sdk/​src/​core/​mxc_telemetry/​mod.rs	Emit Start/End events and session fields.
src/​mxc-sdk/​src/​core/​mxc_engine/​state_aware.rs	Instrument lifecycle dispatch with session scopes and Start events.
src/​mxc-sdk/​src/​core/​mxc_engine/​mod.rs	Retain streaming-handle attribution during operations and cleanup.
src/​mxc-sdk/​src/​core/​mxc_common/​telemetry/​session.rs	Add scoped session and learning-mode context.
src/​mxc-sdk/​src/​core/​mxc_common/​telemetry/​mod.rs	Integrate Start/End emission and invocation suppression.
src/​mxc-sdk/​src/​core/​mxc_common/​telemetry/​events.rs	Add learning-mode classification and event test support.
src/​mxc-sdk/​src/​core/​mxc_common/​telemetry/​correlation_state.rs	Persist late-enabled lifecycle roots and clear stale state.
src/​mxc-sdk/​src/​core/​mxc_common/​telemetry/​consent_prompt.rs	Update disclosure text assertions.
src/​mxc-sdk/​src/​core/​mxc_common/​process_util.rs	Extract Windows parent-process and image-path helpers.
src/​mxc-sdk/​src/​core/​mxc_common/​models.rs	Centralize learning-mode capability classification.
src/​mxc-sdk/​src/​core/​mxc_common/​diagnostic.rs	Reuse process helpers for local diagnostics.
src/​mxc-sdk/​src/​backends/​process_container/​common/​appcontainer_runner.rs	Reuse the shared capability classifier.
src/​mxc-sdk/​resources/​telemetry/​consent/​en-US.json	Introduce v4 lifecycle-correlation disclosure.
sdk/​dotnet/​Microsoft.Mxc.Sdk.Tests/​MxcTelemetryTests.cs	Align .NET consent tests with v4.
docs/​telemetry.md	Explain execution pairs and lifecycle correlation.
docs/​development/​architecture/​telemetry.md	Document event contracts and correlation persistence.
docs/​development/​architecture/​telemetry-consent-design.md	Document re-consent and collector migration requirements.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Scheduled Windows validation still lacks functional assertions for telemetry correlation through a real cross-process container lifecycle.

1 open issue

• Add cross-process telemetry correlation to scheduled Windows validation · New

2 resolved since last review

• Propagate session context across backend worker threads
• Preserve authorization state when resuming handles with unknown IDs

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Scheduled lifecycle validation has a blocking request defect, while successful real-host ETW validation and Privacy/release-owner review remain outstanding.

2 open issues

• Add required all-allow network policy to IsolationSession provision · New
• Add cross-process telemetry correlation to scheduled Windows validation

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Supported-host lifecycle ETW execution remains unverified, and the revised disclosure still requires Privacy and release-owner review.

1 open issue

• Add cross-process telemetry correlation to scheduled Windows validation

1 resolved since last review

• Add required all-allow network policy to IsolationSession provision

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Supported-host lifecycle ETW execution evidence remains outstanding, and the revised disclosure requires Privacy and release-owner review before shipping.

1 open issue

• Add cross-process telemetry correlation to scheduled Windows validation

_{Review effort: Balanced}

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Richie Gomez (richiemsft)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

🔵 Needs a closer look

Privacy-sensitive correlation changes still need real-host lifecycle ETW validation evidence and Privacy/release-owner review of the updated disclosure.

No open issues 🎉

1 resolved since last review

• Add cross-process telemetry correlation to scheduled Windows validation

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Development telemetry activates despite disabled telemetry and denied consent, contradicting the linked opt-in requirements.

9 open issues

• Account for ineligible builds in worker regression test
• Make telemetry CI assertions eligibility-aware
• Make critical_events available on non-Windows targets
• Debug build detection disables telemetry keyword tests in Windows CI
• Add CI coverage for Required-collection consent and policy gates
• Isolate dry-run telemetry state from real user data
• Restore saved session before authorizing panic and control events
• Preserve opt-in consent checks for development telemetry
• Trace assertions do not verify backend event session IDs

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The development telemetry path collects despite disabled telemetry or withdrawn consent, contradicting the linked opt-in requirements, while tests leave session attribution and consent/policy collection gates insufficiently covered.

4 open issues

• Debug build detection disables telemetry keyword tests in Windows CI
• Add CI coverage for Required-collection consent and policy gates
• Preserve opt-in consent checks for development telemetry
• Trace assertions do not verify backend event session IDs

5 resolved since last review

• Account for ineligible builds in worker regression test
• Make telemetry CI assertions eligibility-aware
• Make critical_events available on non-Windows targets
• Isolate dry-run telemetry state from real user data
• Restore saved session before authorizing panic and control events

_{Review effort: Balanced}

[Richie Gomez (richiemsft)]

Thanks, both addressed in e5d4309.

1. Correlation store writes on dry run / rejected calls. load_and_prune no longer writes on a miss. A phase with no record uses a fresh root without storing it. A new on_lifecycle_outcome persists that root only after a successful, non-dry-run start, exec or stop (envelope, typed, streaming and wxc-exec paths), using persist_if_absent under the store lock so a concurrent request's root wins. New regressions: an empty store stays empty for dry-run and rejected start/exec/stop, and a successful late-enabled phase persists a root that later phases reuse. I also updated the existing tests that relied on write-on-miss.

2. telemetry.md consent wording. Both passages now identify consent resource v4 as the current, privacy-reviewed wording. They say it governs only the consent-gated optional events of the published contracts. The four development-contract events (MXC.ExecutionStart, MXC.ExecutionEnd, MXC.Error, MXC.ProcessExited) are deliberately collected without MXC consent under the approved Required-telemetry design, subject only to build eligibility and administrative policy. Consent enforcement for the published contracts is unchanged.

[Copilot]

🟡 Changes recommended

Telemetry collection bypasses explicit opt-in and consent requirements without an approved scope change, and concurrent invocations can emit mismatched session IDs.

5 open issues

• Debug build detection disables telemetry keyword tests in Windows CI
• Coordinate session IDs across concurrent telemetry starts · New
• Add CI coverage for Required-collection consent and policy gates
• Preserve opt-in consent checks for development telemetry
• Trace assertions do not verify backend event session IDs

_{Review effort: Balanced}

[Richie Gomez (richiemsft)]

Addressed the remaining Copilot review comments.

Fixed in 393b6c4, e5d4309 and e8a1a78:

• Unix critical_events gating, the tests that assumed eligibility, and session-before-authorization ordering for panic/cancellation.
• Lifecycle correlation is now persisted only after a successful operation (e5d4309).
• New fresh-thread regression: a development-contract End emitted out of band follows the Required gate without any authorization override.
• The ETW assertions now require every backend event (PolicyHash, ProcessExited, Error) to carry the ExecutionStart session ID, with negative controls.
• New live ETW cases in run_telemetry_etw_smoke_test.ps1 -CheckKeywords: development requests emit with consent denied/absent and policy absent/1/3, and emit nothing for policy 0 or an unsupported value. Published requests still emit nothing without consent.

Design thread (#1588/#1517): the Required-only model for the development contract is intentional and was approved by Privacy. Published contracts keep the consent-gated model.

Not run: the live ETW script needs Administrator and was not run locally; it runs in the Windows CI job. startup_sweep_bounds_stale_deletions failed once under parallel load and passes alone (timing flake).

[Copilot]

🔵 Needs a closer look

Development telemetry bypasses linked opt-in and consent requirements without a linked approved revision, and concurrent first executions can emit inconsistent correlation IDs.

4 open issues

• Debug build detection disables telemetry keyword tests in Windows CI
• Coordinate session IDs across concurrent telemetry starts
• Add CI coverage for Required-collection consent and policy gates
• Preserve opt-in consent checks for development telemetry

1 resolved since last review

• Trace assertions do not verify backend event session IDs

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Telemetry collection bypasses consent and request opt-in gates despite linked requirements explicitly prohibiting opt-out bypass, so the conflicting design must be reconciled or the gates retained.

2 open issues

• Add CI coverage for Required-collection consent and policy gates
• Preserve opt-in consent checks for development telemetry

2 resolved since last review

• Debug build detection disables telemetry keyword tests in Windows CI
• Coordinate session IDs across concurrent telemetry starts

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Human assessment is needed because the ETW keyword check can pass CI despite a failed positive ProcessContainer request, leaving successful ProcessExited behavior unverified.

No open issues 🎉

4 resolved since last review

• Debug build detection disables telemetry keyword tests in Windows CI
• Coordinate session IDs across concurrent telemetry starts
• Add CI coverage for Required-collection consent and policy gates
• Preserve opt-in consent checks for development telemetry

_{Review effort: Balanced}

[Gudge (MGudgin)]

Both findings from my previous review are addressed: lifecycle correlation records are persisted only after successful non-dry-run operations, and both canonical consent passages now identify resource v4. Verified at 7e07059. Correlation-store tests: 30 passed; CLI correlation regression: 1 passed. Lifecycle engine suite: 33 passed, with 2 tests blocked during HKCU test-policy setup by sandbox restrictions. Rust source is unchanged since that verification. The updated executor build and dot-sourced ETW assertion harness passed, leaving exit code 0. Live ETW collection was not run locally.