Skip to content

Add generic environment-creation utilities (PEP 723 PR 5a/16) - #1651

Merged
StellaHuang95 merged 3 commits into
microsoft:mainfrom
StellaHuang95:pep723-pr5-create-happy-path
Jul 27, 2026
Merged

Add generic environment-creation utilities (PEP 723 PR 5a/16)#1651
StellaHuang95 merged 3 commits into
microsoft:mainfrom
StellaHuang95:pep723-pr5-create-happy-path

Conversation

@StellaHuang95

@StellaHuang95 StellaHuang95 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Part of #1602 (PEP 723 inline script env support). Design doc: #1601.

Split for review (3 PRs). Reviewers flagged the original PR 5 as too large, so it is split into three stacked PRs grouped by dependency layer:

Applied together the three PRs are byte-for-byte identical to the original single change. Merge order: 5a → 5b → 5c.

Roadmap context

This is the first slice of PR 5 of 16 in the PEP 723 inline-script roadmap. The full plan lives in #1602.

Phase PR Status
Phase 1: Foundation PR 1: cache key hash utility merged (#1634)
PR 2: cache layout + meta.json sidecar merged (#1635)
PR 3: requires-python to interpreter selection merged (#1636)
Phase 2: Manager PR 4: InlineScriptEnvManager skeleton merged (#1610)
PR 5a: generic env-creation utilities this PR (#1651)
PR 5b: inline-script cache + interpreter utilities #1655
PR 5c: create() happy path (manager + wiring) #1656
PR 6: create() uv-install fallback not started (needs 3, 5)
PR 7: persistence with get, set, and Memento not started (needs 4)
PR 8: activation-time discovery not started (needs 2, 4, 7)
Phase 3: Routing PR 9: route PEP 723 scripts to the inline manager not started (needs 4, 7)
PR 10: per-script project registration not started (needs 9)
Phase 4+: UX / lifecycle PRs 11-16 not started

Why this PR

PR 5c implements InlineScriptEnvManager.create(). Before touching the manager, this PR lands the generic, reusable primitives it relies on — a cross-process file lock, a venv Python-path helper, a cancellation-hardened process runner, and two small createWithProgress options. None of this code is inline-script-specific, so it is reviewed on its own.

What this PR adds

Cross-process file lock (src/common/lockfile.apis.ts, new): acquireFileLock uses an atomic mkdir of a <path>.lock directory plus a per-owner marker file, returning AcquiredFileLock { release, retain }. retain() writes a retained marker so a later acquirer fails fast with ELOCKRETAINED instead of waiting out the 5-minute timeout — used when a build is cancelled mid-flight. Distinct error codes (ELOCKED, ELOCKRETAINED, ELOCKORPHANED, ECOMPROMISED, ERETAINFAILED) separate contention from corruption.

Shared getVenvPythonPath (src/common/utils/virtualEnvironment.ts, new): returns Scripts\python.exe on Windows, else bin/python. Replaces an inline copy in venvUtils and is reused by 5b/5c.

Hardened process helper (src/managers/builtin/helpers.ts): runUV and runPython now share one runProcess implementation whose cancellation guards kill() in try/catch and still emits a clean CancellationError if the process errors after a cancel. Per-caller options preserve existing behavior (collectStderr, logPrefix).

venvUtils.ts: createWithProgress gains CreateWithProgressOptions { trackUvEnvironment }, and CreateEnvironmentResult gains pkgInstallationCancelled so a caller can tell cancellation apart from a real install failure. Existing callers are unaffected (both are optional / additive).

Tests

  • lockfile.apis.unit.test.ts — 9 tests: contention, retain/fail-fast, orphaned and compromised locks, and timeout.
  • virtualEnvironment.unit.test.ts — 2 tests for getVenvPythonPath on Windows and POSIX.
  • helpers.cancellation.unit.test.ts — 4 tests for runProcess cancellation safety.
  • venvUtils.createWithProgress.unit.test.ts — 3 tests for trackUvEnvironment and pkgInstallationCancelled.

On this branch alone npm run compile-tests is clean and npm run unittest reports 1447 passing, 0 failing, 4 pending.

User impact

None. These are internal primitives with no new user-visible behavior. The refactors to helpers.ts and venvUtils.ts are behavior-preserving for existing callers.

@StellaHuang95
StellaHuang95 marked this pull request as ready for review July 23, 2026 22:21
@StellaHuang95 StellaHuang95 added the feature-request Request for new features or functionality label Jul 23, 2026
@StellaHuang95
StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from b1e9b95 to f005e16 Compare July 23, 2026 23:06
@StellaHuang95 StellaHuang95 changed the title Add inline-script environment creation happy path (PEP 723 PR 5/16) Add inline-script environment creation happy path - implementation (PEP 723 PR 5/16) Jul 23, 2026
@StellaHuang95
StellaHuang95 marked this pull request as draft July 23, 2026 23:09
@StellaHuang95
StellaHuang95 marked this pull request as ready for review July 23, 2026 23:14
@StellaHuang95
StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from f005e16 to d36561b Compare July 23, 2026 23:30
@StellaHuang95 StellaHuang95 changed the title Add inline-script environment creation happy path - implementation (PEP 723 PR 5/16) Add generic environment-creation utilities (PEP 723 PR 5a/16) Jul 23, 2026
@StellaHuang95
StellaHuang95 marked this pull request as draft July 23, 2026 23:34
@StellaHuang95
StellaHuang95 marked this pull request as ready for review July 23, 2026 23:35
Comment thread src/common/utils/virtualEnvironment.ts
Comment thread src/managers/builtin/helpers.ts Outdated
Comment thread src/common/lockfile.apis.ts Outdated
Comment thread src/common/lockfile.apis.ts Outdated
Comment thread src/common/lockfile.apis.ts
Comment thread src/managers/builtin/venvUtils.ts Outdated
@StellaHuang95
StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from d36561b to 104166d Compare July 24, 2026 00:13
StellaHuang95 and others added 3 commits July 24, 2026 10:28
Cross-process file lock, venv Python-path helper, cancellation-safe process runner, and createWithProgress tracking options that inline-script environment creation builds on.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 39dcc6a3-0fbd-4f36-9d0f-68677de49c27
@StellaHuang95
StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from 0706167 to e78bda6 Compare July 24, 2026 17:28
@StellaHuang95
StellaHuang95 requested a review from edvilme July 24, 2026 17:28

@edvilme edvilme left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM:)

Comment on lines +125 to +127
async function delay(milliseconds: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, milliseconds));
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this could be moved to a more generic location? But it doesn't bother me too much.

});

proc.on('error', (err) => {
if (cancellationRequested) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is cleanup required by this error stage? you went through and confirmed we aren't leaving stuff behind in people's file systems right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change I made only catches proc.kill() throwing so the operation still reports CancellationError because it didn't catch the exception before, these 2 functions don't do cleanups, cleanup belongs to the caller. If you're asking about inline script manager in particular, that will be handled at the inline manager level later.

@StellaHuang95
StellaHuang95 merged commit dda5f69 into microsoft:main Jul 27, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature-request Request for new features or functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants