Skip to content

test(devin-cli): anchor the empty-data-dir fallback at the host home - #4384

Closed
luvs01 wants to merge 1 commit into
lidge-jun:devfrom
luvs01:agent/windows-cli-fixtures-20260912
Closed

test(devin-cli): anchor the empty-data-dir fallback at the host home#4384
luvs01 wants to merge 1 commit into
lidge-jun:devfrom
luvs01:agent/windows-cli-fixtures-20260912

Conversation

@luvs01

@luvs01 luvs01 commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Summary

The Windows lane fails tests/providers/devin-cli-login.test.ts on current dev. The empty-XDG_DATA_HOME case added with #4418 asserts that the resolved credential path starts with /, but the fallback deliberately resolves against the real home directory, and homedir() returns C:\Users\<name> regardless of which platform the resolver is asked about. The assertion therefore only holds when the host is POSIX.

Anchor both branches at homedir() instead. That is the property the test exists to prove — the path is not cwd-relative — and it holds on either host. The win32 branch needs one extra step: it joins with win32 separators, so a POSIX host home such as /Users/runner comes back as \Users\runner and a raw homedir() prefix would not match. The anchor for that branch is normalized with win32.join(homedir()), which is a no-op on Windows and rewrites the separators on macOS and Linux. Production behavior is unchanged.

Scope change from the original PR

This PR originally carried the Devin path-separator fixture repair plus a pnpm shim fixture repair. Both are now obsolete:

  • The Devin separator problem was fixed upstream by 293c37d68 (fix(devin-cli): resolve credential paths for the selected platform), which made the POSIX expectations correct.
  • tests/providers/devin-cli-adapter.test.ts no longer exists; 213065e30 retired the ACP adapter.
  • The pnpm shim fixtures landed in test(update): use host-native pnpm shim fixtures #4379.

Rather than close this and open a third PR for the same file, I rebuilt the branch on dev@dcd13b435 with only the one remaining Windows failure fixed. The head is a single commit; the previous two-commit series is gone.

Verification

  • Reproduced first on current dev: 17 pass, 1 fail — an empty XDG_DATA_HOME or APPDATA does not become a cwd-relative path at tests/providers/devin-cli-login.test.ts:155, expected true, received false.
  • After the change: bun test ./tests/providers/devin-cli-login.test.ts18 pass, 0 fail, 34 assertions, Bun 1.4.2 on Windows.
  • bun run typecheck, bun run structure:check, bun run privacy:scan, git diff --check — all passed.
  • Cross-platform matrix on this fork caught a second host shape: a first attempt that anchored the win32 branch at a raw homedir() passed Windows and Linux but failed the macOS shard, because win32.join had rewritten /Users/runner to \Users\runner. The normalized anchor is verified for both host shapes: \Users\runner\AppData\Roaming\devin\credentials.toml on a POSIX host and C:\Users\runneradmin\AppData\Roaming\devin\credentials.toml on Windows both anchor correctly.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. Not needed for a test-only correction.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. The assertion still proves the anti-cwd-relative property the original hardening added.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • All CI tests are green on my local testing.

  • I pushed my PR to the latest dev commit.

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Summary by CodeRabbit

  • Tests
    • Improved credential-path test coverage across operating systems.
    • Added validation for home-directory fallback paths and Windows-style path normalization.

Current verification (2026-09-13)

Head: 5a57d08, rebased onto dev@17da84f89.
Focused tests, typecheck, structure check and privacy scan passed on this rebased source:
(pass) devin-cli credential path and read bounds > no thrown message repeats the key [0.13ms]

18 pass
0 fail
40 expect() calls
Ran 18 tests across 1 file. [1423.00ms]

The previous green matrix tested an older PR head with the pending #4384 fix added. It is integration evidence only, not a passing full-suite result for this published head. Full CI readiness remains open. The current dev tip has advanced beyond this tested base; further refresh will be coordinated with the shared Windows fixture fix to avoid repeatedly queuing matrices that inherit the same failure.

@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: de4ee62b-3396-4a65-a927-33a6b6c0d4a7

📥 Commits

Reviewing files that changed from the base of the PR and between dcd13b4 and fdba29b.

📒 Files selected for processing (1)
  • tests/providers/devin-cli-login.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

The credential-path tests now use the runtime home directory and normalize Windows separators. This replaces POSIX-specific and fixed-string path assertions.

Changes

Credential path test portability

Layer / File(s) Summary
Platform-aware credential path assertions
tests/providers/devin-cli-login.test.ts
Lines 2–3 add homedir and win32 imports. Lines 154–168 assert paths under the runtime home directory after Windows separator normalization.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to fdba2

The test-only portability fix is supported by the cross-platform validation results and is ready to merge.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: updating the Devin CLI test so the empty-data-directory fallback is anchored at the host home directory. It matches the test-only scope and t…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lidge-jun

Copy link
Copy Markdown
Owner

리뷰 · 우선순위 54 / 80

설명

이 PR은 Windows에서 깨지는 테스트 픽스처만 고칩니다. 프로덕션 코드는 건드리지 않고 tests/providers/devin-cli-adapter.test.ts, tests/providers/devin-cli-login.test.ts, tests/update/update-pnpm.test.ts 세 파일만 바꿉니다. 지금 dev HEAD는 d42a1363d (#4368 Codex 거절 계정이 다음 픽을 이기지 못하게 한 직후, package.json 2.52.0)이고, 이 PR 본문 base는 그보다 한 칸 뒤인 489af939b입니다. 캐시/ Hermes/ accounts 레인과는 겹치지 않는 CI 수리 조각입니다.

무엇이 깨졌냐면, Devin 경로 기대값이 리눅스식 슬래시 문자열(/home/u/.local/bin/devin, /d/devin/credentials.toml)로 박혀 있어서 Windows 호스트의 path.join 결과와 달라지고, pnpm 글로벌 shim 테스트는 POSIX 셸 스크립트에 실행 비트를 심은 뒤 platform: "linux"로 검증해서 NTFS/Windows CI에서 실패합니다. 이 PR은 Devin 쪽을 join(...)로 맞추고, pnpm 쪽은 writeHostShims로 win32면 .cmd/.ps1, 그 외면 실행 비트 있는 shell shim을 만든 뒤 process.platform으로 검증합니다. 절대 credential override, discovery 우선순위, stale shim 거부, group alias는 유지한다고 본문에 적혀 있고, Spark 은퇴(#4334)는 가져오지 않는다고 선을 긋습니다.

같은 날 열린 #4379(test(update): use host-native pnpm shim fixtures)도 tests/update/update-pnpm.test.ts의 host-native shim 헬퍼를 고칩니다. 두 PR이 pnpm 픽스처를 동시에 고치면 나중에 한쪽이 conflict나 중복 머지가 됩니다. Devin 경로 join은 #4379 범위 밖이라 #4384만의 가치가 있고, pnpm 부분은 사실상 형제 수리입니다. draft이고 mergeable은 MERGEABLE이지만 mergeStateStatus는 BLOCKED(체크 대기)입니다. types.ts/config.ts 분리 캠페인과는 무관합니다.

라인 - 이게 무슨 문제다

tests/update/update-pnpm.test.ts / writeHostShims - #4379와 같은 파일·같은 헬퍼 계열을 고칩니다. 둘 다 머지하면 충돌하거나 한쪽이 중복이 됩니다. pnpm 수리는 한쪽으로 모으는 편이 낫습니다.

tests/providers/devin-cli-adapter.test.ts / join("/home/u", ".local", "bin", "devin" - home 인자는 여전히 POSIX 형태 문자열이지만, 기대값만 host separator로 맞춥니다. Windows에서 home 자체가 다른 형태여도 exists 콜백이 같은 문자열을 받으니 테스트 의도는 유지됩니다. 다만 “진짜 Windows home 레이아웃”까지 검증하진 않습니다.

tests/providers/devin-cli-login.test.ts 절대 override /tmp/creds.toml - 리눅스 플랫폼 인자로 돌릴 때 절대 경로 문자열을 그대로 기대합니다. Windows 전용 절대 경로 케이스는 이 diff에 없어서, Devin Windows APPDATA 케이스와 역할이 나뉩니다. 범위는 괜찮습니다.

본문 Verification - 로컬에서 Devin/pnpm 55 passed를 적었지만 hosted Cross-platform CI는 pending입니다. draft 체크리스트도 아직 ready가 아닙니다.

메인테이너의 판단이 필요한 지점

  • pnpm shim 수리를 #4379에 맡기고 이 PR은 Devin join만 남길지, 반대로 이 PR로 pnpm까지 흡수하고 #4379를 닫을지
  • Windows CI 초록을 이 draft 단독으로 볼지, Combo tip에 묶을지
  • Spark/#4334 픽스처를 이 PR에 더 실을지(본문은 의도적으로 제외)

너의 추천
Devin 경로 join은 바로 가져갈 만합니다. pnpm 부분은 #4379와 겹치니 하나를 owner로 정하고 다른 쪽은 close-or-trim 하세요. Cross-platform(특히 Windows) CI가 초록이면 draft 해제 후 머지, 그 전에는 draft 유지. 프로덕션 미변경이라 위험은 낮습니다.

이 댓글은 grok-bot이 작성했습니다

@luvs01
luvs01 force-pushed the agent/windows-cli-fixtures-20260912 branch from 75c7c16 to 4283b95 Compare September 12, 2026 07:28
@luvs01 luvs01 changed the title test: make Devin and pnpm fixtures portable on Windows test: make Devin path fixtures portable on Windows Sep 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@luvs01

luvs01 commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Exact-head CI for 4283b95 completed with failure: https://github.com/luvs01/opencodex/actions/runs/34681186783

The three Devin fixture failures are absent from this run; both retained files also passed locally (29 tests, 95 assertions). Remaining recurring assertion groups correspond to Cline inventories (#4386), injection/journal fixtures (#4380), and pnpm shim fixtures (#4379).

One additional failure is explicitly unresolved: cli-start-journal-order.test.ts, startup preserves only a client journal matching the final committed api key id, exceeded its 30,000 ms budget in Windows job 103520318533. Slow adjacent cases make load sensitivity plausible, but do not establish the cause. My earlier categorization as runner timing was too strong. This is outside the two modified Devin test files; it remains a follow-up observation, not a passing or resolved test. Draft and CI checkboxes remain unchanged.

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (1/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 1/4).

Review readiness checklist

  • ⬜ All CI tests are green on my local testing.
  • ⬜ I pushed my PR to the latest dev commit.
  • ✅ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

1/4 boxes ticked.

This PR stays in draft until every box above is ticked.

@luvs01
luvs01 force-pushed the agent/windows-cli-fixtures-20260912 branch 2 times, most recently from 3314465 to cba37f3 Compare September 12, 2026 23:12
@luvs01 luvs01 changed the title test: make Devin path fixtures portable on Windows test(devin-cli): anchor the empty-data-dir fallback at the host home Sep 12, 2026
@luvs01

luvs01 commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

This PR was rebuilt on dev@dcd13b435 and its scope is now a single test-only commit. The original two-commit series is gone, so the two questions from the earlier review are resolved by construction rather than by argument:

  • The pnpm shim fixtures are no longer touched here. test(update): use host-native pnpm shim fixtures #4379 owns that repair and has landed, so there is nothing left to deduplicate.
  • tests/providers/devin-cli-adapter.test.ts no longer exists. 213065e30 retired the ACP adapter, and 293c37d68 fixed the credential path resolution that the old separator expectations were working around.

What remains is one real Windows failure on current dev. In tests/providers/devin-cli-login.test.ts:155, the empty-XDG_DATA_HOME case asserts resolved.startsWith("/"), but the fallback resolves against the real home directory, and homedir() returns C:\Users\<name> no matter which platform the resolver is asked about. The assertion only holds when the host itself is POSIX. Anchoring both branches at homedir() keeps the property the test exists to prove — the path never becomes cwd-relative — and it holds on either host.

This is worth knowing beyond this PR: the same failure is currently red on the Windows lane of every head rebased onto dcd13b435. A cross-platform matrix on this fork shows (fail) devin-cli credential path and read bounds > an empty XDG_DATA_HOME or APPDATA does not become a cwd-relative path in the windows 1/6 shard of otherwise unrelated heads, for example run 34724696111. A baseline matrix on unmodified dcd13b435 is running at run 34725771988 to record that the failure belongs to dev and not to any individual change.

Verification on this head: reproduced 17 pass / 1 fail on dev, then 18 pass / 0 fail after the change, with bun run typecheck, bun run structure:check and bun run privacy:scan also passing. Production code is untouched.

@luvs01
luvs01 force-pushed the agent/windows-cli-fixtures-20260912 branch from cba37f3 to fdba29b Compare September 12, 2026 23:59
@luvs01

luvs01 commented Sep 13, 2026

Copy link
Copy Markdown
Contributor Author

Correction to the head, and a finding worth recording.

The first version of this fix anchored both branches at a raw homedir(). That passed on Windows and on Linux, but the cross-platform matrix failed the macOS shard on the same test. The cause is that the win32 branch resolves through win32.join, which rewrites a POSIX host home such as /Users/runner into \Users\runner, so a host-shaped prefix never matched there. The Windows lane had hidden this because win32.join is separator-preserving for C:\Users\<name>.

The head is now fdba29bc1, which normalizes the anchor for that branch with win32.join(homedir()) — a no-op on Windows, a separator rewrite on macOS and Linux. Both host shapes are checked directly: \Users\runner\AppData\Roaming\devin\credentials.toml and C:\Users\runneradmin\AppData\Roaming\devin\credentials.toml both anchor correctly, and the file is 18 pass / 0 fail locally with bun run typecheck, bun run structure:check and bun run privacy:scan green.

A full matrix on this exact head is running at run 34726876847, alongside a baseline matrix on unmodified dcd13b435 at run 34725771988 so the Windows failure can be attributed to dev rather than to any open change.

@Ingwannu Ingwannu left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete one-file patch at fdba29b against dcd13b4.

The new assertions match the resolver's actual contract: its fallback uses the host homedir, while the selected platform controls posix/win32 joining. Checking the Windows branch against win32.join(homedir()) accounts for the separator rewrite on a POSIX host; the data-directory suffix and non-cwd-relative checks remain. No production code, timeout, credentials, workflow, or dependency change is included. This is not the older pnpm/retired ACP patch described in the historical reviews.

I approved exact-head Cross-platform CI 34726826365 and React Doctor 34726826617. The description's cba37f3 evidence is an older revision; these approvals and this review bind fdba29b, not that earlier head. The hosted pull_request lane remains read-only and does not select the self-hosted Windows path.

Keep Draft until the final-head runtime checks and checklist are complete. This execution approval is not merge approval and does not assert every downstream PR will become green. I did not execute tests against the local runtime home.

@luvs01
luvs01 marked this pull request as ready for review September 13, 2026 02:15
@luvs01
luvs01 requested a review from lidge-jun as a code owner September 13, 2026 02:15
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-13T02:17:13.092531Z fdba29b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

The fallback resolves against the real home directory, so asserting a leading slash only held on a POSIX host and failed the Windows lane. Anchor both branches at homedir(), and normalize the anchor for the win32 branch because win32.join rewrites a POSIX host home such as /Users/runner to \Users\runner.
@github-actions
github-actions Bot marked this pull request as draft September 13, 2026 02:43
@luvs01
luvs01 force-pushed the agent/windows-cli-fixtures-20260912 branch from fdba29b to 5a57d08 Compare September 13, 2026 03:08
lidge-jun added a commit that referenced this pull request Sep 13, 2026
…4448)

Carry #4384 from fdba29b onto the current
dev tip.

The empty-XDG_DATA_HOME case asserted the resolved path starts with "/", which
is false on a Windows runner: devinCliCredentialsPath picks path.win32 or
path.posix from its platform ARGUMENT, but the fallback joins the HOST
homedir(), so asking about "linux" from Windows yields
C:\Users\<name>/.local/share/devin/credentials.toml. Anchor both crossings at
that home directory instead, normalising through win32.join for the win32
branch so a POSIX host home compares against the same separators.

Anchoring, not a leading slash, is what proves the path is not cwd-relative,
which is the defect the test exists to pin. The surviving endsWith assertions
and the startsWith("devin") === false check still prove it.

Production behaviour is unchanged; this is test-only.

Local product tests, typecheck, build and install: NOT RUN.
Hosted exact-head CI on this PR is the merge proof.

Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
@lidge-jun

Copy link
Copy Markdown
Owner

Landed on dev as 720ea97 via #4448, carried onto the current tip with your Co-authored-by trailer on the squashed commit, and taken verbatim — the win32.join normalisation for the POSIX-host case was the part my own plan had missed. Exact-head CI was green on all 22 checks including Windows. Thanks for the fix. Closing as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants