diff --git a/CHANGELOG.md b/CHANGELOG.md index 779cadb7..fd1ff65b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,72 @@ All notable changes to the Toolpath workspace are documented here. +## Share and resume over object storage — 2026-08-25 + +**`path-cli`** (0.19.0) can write toolpath documents to an S3 bucket, any +S3-compatible endpoint (R2, MinIO, Ceph, B2), or a plain folder — and +read them back. + +```bash +path p export object --input claude-abc --to s3://my-bucket/traces +path p export object --input claude-abc --to ~/Dropbox/toolpath-traces +path p import object s3://my-bucket/traces/2026-08-07-fix-the-parser-claude-abc.json + +path resume s3://my-bucket/traces/2026-08-07-fix-the-parser-claude-abc.json +path resume ~/Dropbox/toolpath-traces # lists the folder, pick one +``` + +**Credentials come from wherever you already keep them.** If you use the +AWS CLI, this needs no configuration: `~/.aws/credentials` and +`~/.aws/config` are read directly, `AWS_PROFILE` and `--profile` select +a profile, and the profile's region is used if you haven't set one. SSO, +`role_arn` chains, and `credential_process` profiles are resolved by +shelling out to `aws configure export-credentials` — the AWS CLI's own +resolver, so refresh and every future profile type stay its problem. + +When an SSO session has expired, `path` offers to run +`aws sso login --profile ` and retries once, rather than making +you go run it and start over. It asks first — that command opens a +browser and waits — and with no terminal to ask, it fails with the exact +command instead of hanging. + +`object_store` alone would have covered only the server cases (EC2, ECS, +EKS); it reads no `~/.aws` because it avoids the AWS SDK. Taking on +`aws-config` to fix that would have meant 31 crates and an MSRV +treadmill — its family requires rustc 1.94.1 against a repo pinned to +1.94.0. + +`path auth s3 login` is the fallback for endpoints AWS tooling doesn't +know about, where a scoped long-lived token is the right answer. It +stores connection settings at `~/.toolpath/s3.json` (0600) — +deliberately not a destination, so one credential serves any number of +buckets — and merges rather than replaces. `path auth s3 status` reports +which credential source actually won, because that's the first question +when an upload fails. + +**Objects are named to be read.** A document lands at +`--.json`, e.g. +`2026-08-07-add-s3-support-claude-6f2a1c9e.json`. Every component is a +pure function of the document, so re-exporting a session that grew +overwrites its own object instead of leaving near-duplicates; the date +is the session's *first* step, so it doesn't move as the conversation +continues. That legibility is also what makes browsing cheap: +`path resume ` lists a bucket, prefix, or folder and offers +a picker built from object names alone — no downloads. A destination +holding one document skips the picker. + +A scheme-less destination is a **local path**; spell a bucket `s3://`. A +*bare relative* value (`my-bucket/traces`) is rejected rather than +quietly creating `./my-bucket/traces` and reporting success. +`memory://` is rejected too: a fresh per-process store, so anything +written there is gone before the command exits. + +Transport is the `object_store` crate, so one code path covers every +backend. The folder backend is what the tests round-trip against, so +export, import, and resume are exercised end to end without a network or +a mock HTTP server. + + ## `path config edit` — 2026-08-14 - **`path-cli`** (0.18.0): new `path config` porcelain command, starting diff --git a/CLAUDE.md b/CLAUDE.md index 51f0f467..05507d31 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -130,6 +130,14 @@ cargo run -p path-cli -- config edit # $VISUAL/$EDITOR on ~/.toolpath/config.to The **cache** at `~/.toolpath/documents/.json` is the single landing zone for every `import` (and for `import pathbase` downloads). Cache id is `-` — e.g. `claude-abc123`, `git-main` (Pathbase paths key on `--`, anon paths on `anon-pathstash-`). Files are `0600`, parent directory `0700`. `$TOOLPATH_CONFIG_DIR` overrides the root. Imports error on cache hit (`--force` overwrites); `--no-cache` sends the JSON to stdout for shell composition. `p cache sync` fills the cache incrementally from the installed agent harnesses (see "Things to know") and always overwrites what it re-derives. +**Object-storage transport** lives in `crates/path-cli/src/store.rs`, which splits *where a document goes* (`Destination`, `ObjectUri`, `ObjectName` — pure URL parsing and naming, no credentials) from *how to reach it* (`S3Settings` at `~/.toolpath/s3.json`). Transport is the `object_store` crate, so one code path serves AWS S3, any S3-compatible endpoint (R2, MinIO, Ceph, B2 — set `endpoint`), and `file://` for a local folder; it's async, so it tunnels through the same `cmd_pathbase::block_on` runtime the Pathbase client uses. `file://` is a first-class destination, not a toy — it's what the tests round-trip against, so export/import/resume are covered with no network and no mock HTTP server. Accepted schemes are deliberately narrower than what `object_store` parses (`s3`, `s3a`, `file`): `http`/`https` belong to Pathbase in the same dispatch, `gs://`/`az://` would need feature flags we don't compile in, and `memory://` is a fresh per-process store whose contents vanish before the command exits. A scheme-less destination is a **local path**, and a *bare relative* one (`my-bucket/traces`) is rejected — it's overwhelmingly a bucket name typed from memory, and resolving it against the cwd would create `./my-bucket/traces` and report success. + +Object names are `--.json` (`store::name_for`): every component is a pure function of the document, so a re-export overwrites its own object rather than leaving near-duplicates (the date is the *earliest* step's, so it doesn't move as the session grows), and the name is legible enough that `Destination::list` + `cmd_resume::pick_from_destination` build picker rows without downloading anything. + +**S3 credentials** are resolved by `crate::aws_creds`, not by `object_store` alone. `object_store` covers the *server* cases (EKS/IRSA web identity, ECS task roles, EC2 instance metadata) and deliberately reads no `~/.aws` at all, because it avoids depending on the AWS SDK — which leaves out how nearly every developer actually has S3 access. `aws_creds` fills that in: static-key profiles are parsed straight out of `~/.aws/credentials` (trivial ini, no deps), and anything else — SSO, `role_arn` chains, `credential_process` — is delegated to `aws configure export-credentials --format process`, which runs the AWS CLI's own resolver. Anyone using SSO already has the CLI (`aws sso login` is how they authenticate), and delegating keeps refresh, cache layout, and future profile types the CLI's problem. An expired SSO session is special-cased because it has one obvious fix: `aws_creds` offers to run `aws sso login --profile ` and retries **once**, prompting first (the command opens a browser and waits, so it must not fire because someone typed `path resume`) and, with no terminal to ask, failing with the exact command rather than blocking. Detection matches the CLI's message, whose wording varies by version. Depending on `aws-config` instead would be 31 crates *and* an MSRV treadmill: its whole family currently requires rustc 1.94.1 while `rust-toolchain.toml` pins 1.94.0. + +Precedence (AWS's own, with our stored settings layered on top): `path auth s3 login` → `--profile` / `$AWS_PROFILE` → `AWS_ACCESS_KEY_ID` → the `[default]` profile → `object_store`'s instance chain. Region falls back to the profile's. `path auth s3 status` prints *which* source won — the first question when an upload fails is always which credential was tried. `AWS_SHARED_CREDENTIALS_FILE` / `AWS_CONFIG_FILE` are honored, which is also how the integration tests stay off a developer's real profiles. + `path auth login` prints `/auth/cli`; the user logs in there and pastes the 8-character code back, which the CLI redeems (`POST /api/v1/auth/cli/redeem`) for a bearer token stored at `~/.toolpath/credentials.json` (`0600`; `$TOOLPATH_CONFIG_DIR` overrides). Server URL comes from `--url`, then `$PATHBASE_URL`, then `https://pathbase.dev`. The redeem endpoint is real but absent from `schema/pathbase-openapi.json` — so the progenitor-derived `pathbase-client` has no `redeem` method; the hand-rolled call in `cmd_pathbase.rs` is the source of truth. ## Key conventions diff --git a/Cargo.lock b/Cargo.lock index 4be9d924..0db24ca7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -151,6 +151,17 @@ version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1685feee7d06d8813fe963f814c5c398d90392b9c3c41e656ac3100d5c334536" +[[package]] +name = "async-trait" +version = "0.1.92" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.4", +] + [[package]] name = "atomic" version = "0.6.1" @@ -269,6 +280,15 @@ dependencies = [ "generic-array", ] +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + [[package]] name = "borrow-or-share" version = "0.2.4" @@ -349,6 +369,17 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" +[[package]] +name = "chacha20" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d524456ba66e72eb8b115ff89e01e497f8e6d11d78b70b1aa13c0fbd97540a81" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "rand_core 0.10.1", +] + [[package]] name = "chrono" version = "0.4.44" @@ -543,6 +574,25 @@ dependencies = [ "libc", ] +[[package]] +name = "cpufeatures" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201" +dependencies = [ + "libc", +] + +[[package]] +name = "crc-fast" +version = "1.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e75b2483e97a5a7da73ac68a05b629f9c53cff58d8ed1c77866079e18b00dba5" +dependencies = [ + "digest 0.10.7", + "spin", +] + [[package]] name = "crossbeam-deque" version = "0.8.7" @@ -608,6 +658,15 @@ dependencies = [ "typenum", ] +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + [[package]] name = "csscolorparser" version = "0.6.2" @@ -806,8 +865,18 @@ version = "0.10.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ - "block-buffer", - "crypto-common", + "block-buffer 0.10.4", + "crypto-common 0.1.7", +] + +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer 0.12.1", + "crypto-common 0.2.2", ] [[package]] @@ -1076,7 +1145,7 @@ version = "0.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ca1f7b0dce1c25254457f2edcfcf611bf207113564b96f90659a3f54506016b2" dependencies = [ - "itertools", + "itertools 0.14.0", "raw-cpuid", ] @@ -1229,6 +1298,7 @@ dependencies = [ "cfg-if", "libc", "r-efi 6.0.0", + "rand_core 0.10.1", "wasip2", "wasip3", ] @@ -1374,6 +1444,21 @@ version = "1.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87" +[[package]] +name = "humantime" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15cdd26707701c53297e2fa6afb323d55fbc1d0810c3aec078ae3ef0424c3c15" + +[[package]] +name = "hybrid-array" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "707114b52a152fa7bdb290cd7cd5912d9467273b6d74e21b8d81aca1f8533f6b" +dependencies = [ + "typenum", +] + [[package]] name = "hyper" version = "1.9.0" @@ -1707,6 +1792,15 @@ dependencies = [ "either", ] +[[package]] +name = "itertools" +version = "0.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b4baf93f58d4425749ca49a51c50ebab072c5df6994d08fed93541c331481dc" +dependencies = [ + "either", +] + [[package]] name = "itoa" version = "1.0.18" @@ -2111,6 +2205,16 @@ dependencies = [ "winapi", ] +[[package]] +name = "md-5" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69b6441f590336821bb897fb28fc622898ccceb1d6cea3fde5ea86b090c4de98" +dependencies = [ + "cfg-if", + "digest 0.11.3", +] + [[package]] name = "memchr" version = "2.8.0" @@ -2384,6 +2488,48 @@ dependencies = [ "memchr", ] +[[package]] +name = "object_store" +version = "0.14.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d354792e39fa5f0009e47623cf8b15b099bf9a652fa55c6f817fe28ac84fea50" +dependencies = [ + "async-trait", + "aws-lc-rs", + "base64", + "bytes", + "chrono", + "crc-fast", + "form_urlencoded", + "futures-channel", + "futures-core", + "futures-util", + "http", + "http-body-util", + "humantime", + "hyper", + "itertools 0.15.0", + "md-5", + "nix 0.31.3", + "parking_lot", + "percent-encoding", + "quick-xml", + "rand 0.10.2", + "reqwest", + "rustls-pki-types", + "serde", + "serde_json", + "serde_urlencoded", + "thiserror 2.0.18", + "tokio", + "tracing", + "url", + "walkdir", + "wasm-bindgen-futures", + "web-time", + "windows-sys 0.61.2", +] + [[package]] name = "once_cell" version = "1.21.4" @@ -2487,7 +2633,7 @@ dependencies = [ [[package]] name = "path-cli" -version = "0.18.0" +version = "0.19.0" dependencies = [ "anyhow", "assert_cmd", @@ -2501,12 +2647,14 @@ dependencies = [ "jaq-json", "jaq-std", "jsonschema", + "object_store", "pathbase-client", "predicates", "rand 0.9.4", "rayon", "regex", "reqwest", + "rpassword", "rusqlite", "serde", "serde_json", @@ -2529,6 +2677,7 @@ dependencies = [ "toolpath-md", "toolpath-opencode", "toolpath-pi", + "url", "uuid", ] @@ -2902,6 +3051,16 @@ dependencies = [ "syn 2.0.117", ] +[[package]] +name = "quick-xml" +version = "0.41.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1" +dependencies = [ + "memchr", + "serde", +] + [[package]] name = "quinn" version = "0.11.9" @@ -2998,6 +3157,17 @@ dependencies = [ "rand_core 0.9.5", ] +[[package]] +name = "rand" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80" +dependencies = [ + "chacha20", + "getrandom 0.4.2", + "rand_core 0.10.1", +] + [[package]] name = "rand_chacha" version = "0.9.0" @@ -3023,6 +3193,12 @@ dependencies = [ "getrandom 0.3.4", ] +[[package]] +name = "rand_core" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" + [[package]] name = "ratatui" version = "0.30.0" @@ -3047,7 +3223,7 @@ dependencies = [ "compact_str", "hashbrown 0.16.1", "indoc", - "itertools", + "itertools 0.14.0", "kasuari", "lru", "strum", @@ -3099,7 +3275,7 @@ dependencies = [ "hashbrown 0.16.1", "indoc", "instability", - "itertools", + "itertools 0.14.0", "line-clipping", "ratatui-core", "strum", @@ -3322,6 +3498,27 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "rpassword" +version = "7.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2da316a15f47e3d053de9cb2c439650bd8fa4aaeb9365f2e5f27f492ff73c196" +dependencies = [ + "libc", + "rtoolbox", + "windows-sys 0.61.2", +] + +[[package]] +name = "rtoolbox" +version = "0.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "50a0e551c1e27e1731aba276dbeaeac73f53c7cd34d1bda485d02bd1e0f36844" +dependencies = [ + "libc", + "windows-sys 0.59.0", +] + [[package]] name = "rusqlite" version = "0.32.1" @@ -3664,8 +3861,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba" dependencies = [ "cfg-if", - "cpufeatures", - "digest", + "cpufeatures 0.2.17", + "digest 0.10.7", ] [[package]] @@ -3675,8 +3872,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ "cfg-if", - "cpufeatures", - "digest", + "cpufeatures 0.2.17", + "digest 0.10.7", ] [[package]] @@ -3841,6 +4038,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "spin" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3" + [[package]] name = "stable_deref_trait" version = "1.2.1" @@ -3908,6 +4111,17 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "syn" +version = "3.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + [[package]] name = "sync_wrapper" version = "1.0.2" @@ -4436,9 +4650,21 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" dependencies = [ "pin-project-lite", + "tracing-attributes", "tracing-core", ] +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + [[package]] name = "tracing-core" version = "0.1.36" @@ -4609,7 +4835,7 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "16b380a1238663e5f8a691f9039c73e1cdae598a30e9855f541d29b08b53e9a5" dependencies = [ - "itertools", + "itertools 0.14.0", "unicode-segmentation", "unicode-width", ] @@ -5125,6 +5351,15 @@ dependencies = [ "windows-targets 0.52.6", ] +[[package]] +name = "windows-sys" +version = "0.59.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" +dependencies = [ + "windows-targets 0.52.6", +] + [[package]] name = "windows-sys" version = "0.60.2" diff --git a/Cargo.toml b/Cargo.toml index b57ad62b..5766959e 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -37,7 +37,7 @@ toolpath-github = { version = "0.6.0", path = "crates/toolpath-github" } toolpath-dot = { version = "0.5.0", path = "crates/toolpath-dot" } toolpath-md = { version = "0.7.0", path = "crates/toolpath-md" } toolpath-pi = { version = "0.6.1", path = "crates/toolpath-pi" } -path-cli = { version = "0.18.0", path = "crates/path-cli" } +path-cli = { version = "0.19.0", path = "crates/path-cli" } pathbase-client = { version = "0.2.0", path = "crates/pathbase-client" } reqwest = { version = "0.13", default-features = false, features = ["blocking", "json", "rustls"] } diff --git a/crates/path-cli/Cargo.toml b/crates/path-cli/Cargo.toml index 27bda67b..b650316b 100644 --- a/crates/path-cli/Cargo.toml +++ b/crates/path-cli/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "path-cli" -version = "0.18.0" +version = "0.19.0" edition.workspace = true license.workspace = true repository = "https://github.com/empathic/toolpath" @@ -43,6 +43,13 @@ jaq-json = "2.0.1" figment = { version = "0.10", features = ["env"] } [target.'cfg(not(target_os = "emscripten"))'.dependencies] +# Share/resume over object storage. `aws` covers every S3-compatible +# endpoint; the default `fs` feature gives `file://` for local +# destinations and for the round-trip tests. +object_store = { version = "0.14.1", features = ["aws"] } +url = "2" +# No-echo prompt for the S3 secret access key in `path auth s3 login`. +rpassword = "7" toolpath-claude = { workspace = true, features = ["watcher"] } toolpath-gemini = { workspace = true, features = ["watcher"] } toolpath-codex = { workspace = true } diff --git a/crates/path-cli/src/aws_creds.rs b/crates/path-cli/src/aws_creds.rs new file mode 100644 index 00000000..47f7f605 --- /dev/null +++ b/crates/path-cli/src/aws_creds.rs @@ -0,0 +1,841 @@ +//! Where S3 credentials actually come from. +//! +//! `object_store` resolves static keys, EKS/IRSA web identity, ECS task +//! roles, and EC2 instance metadata — the *server* cases. It reads no +//! `~/.aws/credentials`, no `~/.aws/config`, no `AWS_PROFILE`, and no +//! SSO, because it deliberately avoids depending on the AWS SDK. That +//! leaves out how nearly every developer actually has S3 access on a +//! laptop. +//! +//! This module fills that in without taking on the SDK: +//! +//! - **Static-key profiles** are read directly. `~/.aws/credentials` is +//! a trivial ini file and the keys are right there. +//! - **Everything else** — SSO, `role_arn` chains, `credential_process` +//! — is delegated to `aws configure export-credentials`, which runs +//! the AWS CLI's own resolver and hands back concrete keys. Anyone +//! using SSO already has the CLI, since `aws sso login` is how they +//! authenticate; and delegating means refresh, cache layout, and +//! every future profile type stay the CLI's problem, not ours. +//! +//! An expired SSO session is the one failure with an obvious next step, +//! so it gets one: we offer to run `aws sso login` and retry, rather +//! than making the user go do it and start the command over. Offer, not +//! do — that command opens a browser and waits, which shouldn't happen +//! because someone typed `path resume`. With no terminal to ask (CI), +//! it fails with the exact command instead of blocking on a prompt +//! nobody will answer. +//! +//! Depending on `aws-config` instead would be 31 crates, and its whole +//! family currently requires rustc 1.94.1 while this repo pins 1.94.0 +//! — an MSRV treadmill on a toolchain we pin exactly. + +#![cfg(not(target_os = "emscripten"))] + +use anyhow::{Context, Result, bail}; +use std::collections::HashMap; +use std::path::{Path, PathBuf}; + +/// Concrete keys, plus where they came from. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct Credentials { + pub access_key_id: String, + pub secret_access_key: String, + pub session_token: Option, +} + +/// Which resolution step supplied the credentials. +/// +/// Carried all the way to `path auth s3 status`, because the first +/// question when a share fails is always *which* credential was used — +/// and "the one you configured" and "whatever the AWS CLI resolved" are +/// very different debugging stories. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum Source { + /// Stored by `path auth s3 login`. + Stored, + /// `AWS_ACCESS_KEY_ID` and friends. + Environment, + /// Static keys in `~/.aws/credentials`. + Profile { name: String, file: PathBuf }, + /// Resolved by the AWS CLI (SSO, assume-role, credential_process). + AwsCli { name: String }, + /// Nothing local; `object_store` will try instance metadata, ECS, + /// and web identity on its own. + InstanceChain, +} + +impl std::fmt::Display for Source { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Source::Stored => f.write_str("stored by `path auth s3 login`"), + Source::Environment => f.write_str("AWS_ACCESS_KEY_ID (environment)"), + Source::Profile { name, file } => { + write!(f, "profile `{name}` in {}", file.display()) + } + Source::AwsCli { name } => { + write!(f, "profile `{name}` via the AWS CLI") + } + Source::InstanceChain => { + f.write_str("none found locally — will try the EC2/ECS/EKS credential chain") + } + } + } +} + +/// The outcome of resolution: keys when we found them, always a source. +#[derive(Debug, Clone)] +pub(crate) struct Resolved { + pub credentials: Option, + pub source: Source, + /// Region the profile declares, if any — `~/.aws/config` is the + /// natural place for it and re-typing it would be silly. + pub region: Option, +} + +/// Everything resolution reads, injected so tests need no `$HOME` +/// games and no real AWS CLI. +pub(crate) struct Env<'a> { + pub home: Option, + pub var: &'a dyn Fn(&str) -> Option, + /// Runs `aws configure export-credentials --profile `, + /// returning its stdout. + pub aws_cli: &'a dyn Fn(&str) -> Result, + /// Runs `aws sso login --profile `. Interactive: it opens a + /// browser and waits. + pub sso_login: &'a dyn Fn(&str) -> Result<()>, + /// Asks the user a yes/no question. `false` when there's nobody to + /// ask — a CI run must never block on a prompt. + pub confirm: &'a dyn Fn(&str) -> bool, +} + +impl Env<'_> { + fn credentials_file(&self) -> Option { + let explicit = self + .var("AWS_SHARED_CREDENTIALS_FILE") + .filter(|v| !v.trim().is_empty()); + match explicit { + Some(p) => Some(PathBuf::from(p)), + None => self.home.as_ref().map(|h| h.join(".aws/credentials")), + } + } + + fn config_file(&self) -> Option { + let explicit = self.var("AWS_CONFIG_FILE").filter(|v| !v.trim().is_empty()); + match explicit { + Some(p) => Some(PathBuf::from(p)), + None => self.home.as_ref().map(|h| h.join(".aws/config")), + } + } + + fn var(&self, key: &str) -> Option { + (self.var)(key) + } +} + +/// Resolve S3 credentials, following AWS's own precedence with our +/// stored settings layered on top. +/// +/// 1. `path auth s3 login` — an explicit local choice beats ambient +/// state, and it's the only way to configure a non-AWS endpoint. +/// 2. `--profile` / `AWS_PROFILE` — naming a profile is also explicit. +/// 3. `AWS_ACCESS_KEY_ID` — what CI sets. +/// 4. The `[default]` profile. +/// 5. Nothing: leave it to `object_store`'s instance chain. +pub(crate) fn resolve( + stored: Option, + profile_flag: Option<&str>, + env: &Env<'_>, +) -> Result { + if let Some(credentials) = stored { + return Ok(Resolved { + credentials: Some(credentials), + source: Source::Stored, + region: None, + }); + } + + let named = profile_flag + .map(str::to_string) + .or_else(|| env.var("AWS_PROFILE")) + .filter(|p| !p.trim().is_empty()); + + if let Some(name) = &named { + return from_profile(name, env).with_context(|| format!("profile `{name}`")); + } + + if let (Some(key), Some(secret)) = ( + env.var("AWS_ACCESS_KEY_ID") + .filter(|v| !v.trim().is_empty()), + env.var("AWS_SECRET_ACCESS_KEY") + .filter(|v| !v.trim().is_empty()), + ) { + return Ok(Resolved { + credentials: Some(Credentials { + access_key_id: key, + secret_access_key: secret, + session_token: env.var("AWS_SESSION_TOKEN").filter(|v| !v.is_empty()), + }), + source: Source::Environment, + region: env + .var("AWS_REGION") + .or_else(|| env.var("AWS_DEFAULT_REGION")), + }); + } + + // A `[default]` profile is only used when it exists; its absence is + // not an error, it just means we fall through to the chain. + if profile_exists("default", env) { + return from_profile("default", env).context("profile `default`"); + } + + Ok(Resolved { + credentials: None, + source: Source::InstanceChain, + region: None, + }) +} + +fn profile_exists(name: &str, env: &Env<'_>) -> bool { + let in_credentials = env + .credentials_file() + .and_then(|p| read_ini(&p).ok()) + .is_some_and(|ini| ini.contains_key(name)); + let in_config = env + .config_file() + .and_then(|p| read_ini(&p).ok()) + .is_some_and(|ini| ini.contains_key(&config_section(name)) || ini.contains_key(name)); + in_credentials || in_config +} + +/// Read one profile: static keys if it has them, otherwise the AWS CLI. +fn from_profile(name: &str, env: &Env<'_>) -> Result { + let credentials_path = env.credentials_file(); + let region = profile_region(name, env); + + if let Some(path) = &credentials_path + && let Ok(ini) = read_ini(path) + && let Some(section) = ini.get(name) + && let (Some(key), Some(secret)) = ( + section.get("aws_access_key_id"), + section.get("aws_secret_access_key"), + ) + { + return Ok(Resolved { + credentials: Some(Credentials { + access_key_id: key.clone(), + secret_access_key: secret.clone(), + session_token: section.get("aws_session_token").cloned(), + }), + source: Source::Profile { + name: name.to_string(), + file: path.clone(), + }, + region, + }); + } + + if !profile_exists(name, env) { + bail!( + "no such profile. Check `aws configure list-profiles`, or run \ + `path auth s3 login` to store keys directly." + ); + } + + // The profile exists but carries no static keys: SSO, an assume-role + // chain, or credential_process. The AWS CLI already knows how to + // resolve all of those, including refresh. + let raw = match (env.aws_cli)(name) { + Ok(raw) => raw, + // An expired SSO session is the one failure with an obvious + // next step, and making the user go run it themselves and start + // over is a pointless round trip — so offer to run it here. + // Offer, not do: `aws sso login` opens a browser and waits, and + // that shouldn't happen because someone typed `path resume`. + Err(e) if is_expired_sso(&e) => { + let cmd = format!("aws sso login --profile {name}"); + if !(env.confirm)(&format!( + "The SSO session for profile `{name}` has expired. Run `{cmd}` now?" + )) { + bail!("the SSO session has expired.\n\nRun `{cmd}`, then try again."); + } + (env.sso_login)(name)?; + // Exactly one retry. If a fresh login still doesn't yield + // credentials, looping won't help and the real error is + // whatever comes back now. + (env.aws_cli)(name).context("after `aws sso login`")? + } + Err(e) => return Err(e), + }; + let creds = parse_export_credentials(&raw)?; + Ok(Resolved { + credentials: Some(creds), + source: Source::AwsCli { + name: name.to_string(), + }, + region, + }) +} + +fn profile_region(name: &str, env: &Env<'_>) -> Option { + let ini = read_ini(&env.config_file()?).ok()?; + // `~/.aws/config` spells non-default profiles `[profile name]`. + let section = ini.get(&config_section(name)).or_else(|| ini.get(name))?; + section.get("region").cloned() +} + +fn config_section(name: &str) -> String { + if name == "default" { + name.to_string() + } else { + format!("profile {name}") + } +} + +/// `aws configure export-credentials --format process` output. +fn parse_export_credentials(raw: &str) -> Result { + let v: serde_json::Value = + serde_json::from_str(raw.trim()).context("the AWS CLI returned output that isn't JSON")?; + let field = |k: &str| v.get(k).and_then(|x| x.as_str()).map(str::to_string); + let (Some(access_key_id), Some(secret_access_key)) = + (field("AccessKeyId"), field("SecretAccessKey")) + else { + bail!("the AWS CLI returned no credentials"); + }; + Ok(Credentials { + access_key_id, + secret_access_key, + session_token: field("SessionToken"), + }) +} + +/// True when the AWS CLI failed because an SSO session needs renewing. +/// +/// Matched on the message because the CLI reports it as a plain +/// non-zero exit, and its wording varies by version — "Token has +/// expired and refresh failed", "does not exist", and a direct +/// instruction to run `aws sso login` have all been observed. Requiring +/// `sso` alongside keeps unrelated failures out. +fn is_expired_sso(err: &anyhow::Error) -> bool { + let msg = err.to_string().to_ascii_lowercase(); + msg.contains("sso") + && (msg.contains("expired") + || msg.contains("does not exist") + || msg.contains("refresh failed") + || msg.contains("sso login")) +} + +/// Run `aws sso login`, inheriting stdio so its device code and browser +/// prompt reach the user. Kept behind [`Env::sso_login`] so tests don't. +pub(crate) fn run_sso_login(profile: &str) -> Result<()> { + eprintln!("Running `aws sso login --profile {profile}`…"); + let status = std::process::Command::new("aws") + .args(["sso", "login", "--profile", profile]) + .status() + .context("running `aws sso login`")?; + if !status.success() { + bail!("`aws sso login --profile {profile}` did not complete"); + } + Ok(()) +} + +/// Ask a yes/no question, defaulting to yes. +/// +/// Returns `false` without asking when either stream isn't a terminal: +/// a CI run must fail with instructions rather than block forever on a +/// prompt nobody will answer. The question goes to stderr so stdout +/// stays clean for piping. +pub(crate) fn confirm_on_tty(question: &str) -> bool { + use std::io::{BufRead, IsTerminal, Write}; + if !std::io::stdin().is_terminal() || !std::io::stderr().is_terminal() { + return false; + } + eprint!("{question} [Y/n] "); + let _ = std::io::stderr().flush(); + let mut line = String::new(); + if std::io::stdin().lock().read_line(&mut line).is_err() { + return false; + } + let answer = line.trim().to_ascii_lowercase(); + answer.is_empty() || answer == "y" || answer == "yes" +} + +/// Run the real AWS CLI. Kept behind [`Env::aws_cli`] so tests don't. +pub(crate) fn run_aws_cli(profile: &str) -> Result { + let out = std::process::Command::new("aws") + .args(["configure", "export-credentials", "--profile", profile]) + .args(["--format", "process"]) + .output() + .map_err(|e| match e.kind() { + std::io::ErrorKind::NotFound => anyhow::anyhow!( + "this profile needs the AWS CLI to resolve (SSO, assume-role, or \ + credential_process), but `aws` isn't on PATH. Install it, or run \ + `path auth s3 login` to store keys directly." + ), + _ => anyhow::anyhow!("running `aws configure export-credentials`: {e}"), + })?; + if !out.status.success() { + let stderr = String::from_utf8_lossy(&out.stderr); + bail!( + "the AWS CLI could not resolve this profile: {}", + stderr.trim() + ); + } + Ok(String::from_utf8_lossy(&out.stdout).into_owned()) +} + +// ── Minimal ini ───────────────────────────────────────────────────────── + +/// Parse the subset of ini that AWS config files actually use: +/// `[section]` headers, `key = value` pairs, `#`/`;` comments. +/// +/// Nested sub-sections (the `[services]` style) are ignored rather than +/// mis-parsed — nothing we read lives in one. +fn read_ini(path: &Path) -> Result>> { + let text = std::fs::read_to_string(path)?; + Ok(parse_ini(&text)) +} + +fn parse_ini(text: &str) -> HashMap> { + let mut out: HashMap> = HashMap::new(); + let mut current: Option = None; + for line in text.lines() { + // Indented lines continue a nested sub-section; skip them. + if line.starts_with(char::is_whitespace) { + continue; + } + let line = line.trim(); + if line.is_empty() || line.starts_with('#') || line.starts_with(';') { + continue; + } + if let Some(name) = line.strip_prefix('[').and_then(|l| l.strip_suffix(']')) { + let name = name.trim().to_string(); + out.entry(name.clone()).or_default(); + current = Some(name); + continue; + } + if let (Some(section), Some((key, value))) = (current.as_ref(), line.split_once('=')) { + out.entry(section.clone()) + .or_default() + .insert(key.trim().to_ascii_lowercase(), value.trim().to_string()); + } + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + + /// An `Env` over an in-memory fake `~/.aws` and a stub AWS CLI, so + /// nothing here touches the developer's real credentials. + struct Fake { + dir: tempfile::TempDir, + vars: HashMap, + cli_result: std::cell::RefCell>, + cli_calls: std::cell::RefCell>, + /// What `aws_cli` returns on the *second* call, once a login has + /// happened. `None` keeps returning `cli_result`. + cli_after_login: std::cell::RefCell>, + login_calls: std::cell::RefCell>, + login_fails: std::cell::RefCell, + answer: std::cell::RefCell, + questions: std::cell::RefCell>, + } + + impl Fake { + fn new() -> Self { + let dir = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(dir.path().join(".aws")).unwrap(); + Fake { + dir, + vars: HashMap::new(), + cli_result: std::cell::RefCell::new(Err("aws CLI not stubbed".into())), + cli_calls: std::cell::RefCell::new(Vec::new()), + cli_after_login: std::cell::RefCell::new(None), + login_calls: std::cell::RefCell::new(Vec::new()), + login_fails: std::cell::RefCell::new(false), + answer: std::cell::RefCell::new(true), + questions: std::cell::RefCell::new(Vec::new()), + } + } + fn credentials(self, body: &str) -> Self { + std::fs::write(self.dir.path().join(".aws/credentials"), body).unwrap(); + self + } + fn config(self, body: &str) -> Self { + std::fs::write(self.dir.path().join(".aws/config"), body).unwrap(); + self + } + fn var(mut self, k: &str, v: &str) -> Self { + self.vars.insert(k.to_string(), v.to_string()); + self + } + fn cli(self, out: &str) -> Self { + *self.cli_result.borrow_mut() = Ok(out.to_string()); + self + } + fn cli_err(self, msg: &str) -> Self { + *self.cli_result.borrow_mut() = Err(msg.to_string()); + self + } + fn cli_after_login(self, out: &str) -> Self { + *self.cli_after_login.borrow_mut() = Some(out.to_string()); + self + } + fn login_fails(self) -> Self { + *self.login_fails.borrow_mut() = true; + self + } + fn declines(self) -> Self { + *self.answer.borrow_mut() = false; + self + } + fn resolve(&self, stored: Option, profile: Option<&str>) -> Result { + let var = |k: &str| self.vars.get(k).cloned(); + let cli = |name: &str| { + self.cli_calls.borrow_mut().push(name.to_string()); + // After a login, return the post-login result if one was + // staged — that's what a successful refresh looks like. + if !self.login_calls.borrow().is_empty() + && let Some(out) = self.cli_after_login.borrow().clone() + { + return Ok(out); + } + self.cli_result + .borrow() + .clone() + .map_err(|e| anyhow::anyhow!(e)) + }; + let login = |name: &str| { + self.login_calls.borrow_mut().push(name.to_string()); + if *self.login_fails.borrow() { + anyhow::bail!("`aws sso login --profile {name}` did not complete"); + } + Ok(()) + }; + let confirm = |q: &str| { + self.questions.borrow_mut().push(q.to_string()); + *self.answer.borrow() + }; + resolve( + stored, + profile, + &Env { + home: Some(self.dir.path().to_path_buf()), + var: &var, + aws_cli: &cli, + sso_login: &login, + confirm: &confirm, + }, + ) + } + } + + const STATIC_PROFILE: &str = "\ +[default] +aws_access_key_id = AKIADEFAULT +aws_secret_access_key = defaultsecret + +[work] +aws_access_key_id = AKIAWORK +aws_secret_access_key = worksecret +aws_session_token = worktoken +"; + + #[test] + fn a_default_profile_is_used_with_no_configuration_at_all() { + // The whole point: someone who has run `aws configure` gets S3 + // access without telling us anything. + let f = Fake::new().credentials(STATIC_PROFILE); + let r = f.resolve(None, None).unwrap(); + let c = r.credentials.unwrap(); + assert_eq!(c.access_key_id, "AKIADEFAULT"); + assert!(matches!(r.source, Source::Profile { .. })); + } + + #[test] + fn an_explicit_profile_beats_the_default() { + let f = Fake::new().credentials(STATIC_PROFILE); + let r = f.resolve(None, Some("work")).unwrap(); + let c = r.credentials.unwrap(); + assert_eq!(c.access_key_id, "AKIAWORK"); + assert_eq!(c.session_token.as_deref(), Some("worktoken")); + } + + #[test] + fn aws_profile_env_selects_the_profile() { + let f = Fake::new() + .credentials(STATIC_PROFILE) + .var("AWS_PROFILE", "work"); + let r = f.resolve(None, None).unwrap(); + assert_eq!(r.credentials.unwrap().access_key_id, "AKIAWORK"); + } + + #[test] + fn the_profile_flag_beats_aws_profile() { + let f = Fake::new() + .credentials(STATIC_PROFILE) + .var("AWS_PROFILE", "work"); + let r = f.resolve(None, Some("default")).unwrap(); + assert_eq!(r.credentials.unwrap().access_key_id, "AKIADEFAULT"); + } + + #[test] + fn stored_settings_beat_everything_ambient() { + // Running `path auth s3 login` is an explicit local choice, and + // it's the only way to reach a non-AWS endpoint. + let f = Fake::new() + .credentials(STATIC_PROFILE) + .var("AWS_ACCESS_KEY_ID", "AKIAENV") + .var("AWS_SECRET_ACCESS_KEY", "envsecret"); + let stored = Credentials { + access_key_id: "AKIASTORED".into(), + secret_access_key: "storedsecret".into(), + session_token: None, + }; + let r = f.resolve(Some(stored), None).unwrap(); + assert_eq!(r.credentials.unwrap().access_key_id, "AKIASTORED"); + assert_eq!(r.source, Source::Stored); + } + + #[test] + fn env_keys_beat_the_default_profile() { + // AWS's own precedence, and what CI sets. + let f = Fake::new() + .credentials(STATIC_PROFILE) + .var("AWS_ACCESS_KEY_ID", "AKIAENV") + .var("AWS_SECRET_ACCESS_KEY", "envsecret"); + let r = f.resolve(None, None).unwrap(); + assert_eq!(r.credentials.unwrap().access_key_id, "AKIAENV"); + assert_eq!(r.source, Source::Environment); + } + + #[test] + fn nothing_configured_defers_to_the_instance_chain() { + // On a server this is the correct answer, not a failure. + let f = Fake::new(); + let r = f.resolve(None, None).unwrap(); + assert!(r.credentials.is_none()); + assert_eq!(r.source, Source::InstanceChain); + } + + #[test] + fn an_sso_profile_is_resolved_through_the_aws_cli() { + // No static keys in the file — exactly what `aws sso login` + // leaves behind. The CLI knows how to turn it into keys. + let f = Fake::new() + .config("[profile sso-work]\nsso_session = corp\nsso_account_id = 1234\n") + .cli(r#"{"Version":1,"AccessKeyId":"ASIASSO","SecretAccessKey":"ssosecret","SessionToken":"ssotoken"}"#); + let r = f.resolve(None, Some("sso-work")).unwrap(); + let c = r.credentials.unwrap(); + assert_eq!(c.access_key_id, "ASIASSO"); + assert_eq!(c.session_token.as_deref(), Some("ssotoken")); + assert_eq!( + r.source, + Source::AwsCli { + name: "sso-work".into() + } + ); + assert_eq!(*f.cli_calls.borrow(), vec!["sso-work".to_string()]); + } + + #[test] + fn a_static_profile_never_shells_out() { + let f = Fake::new().credentials(STATIC_PROFILE); + f.resolve(None, Some("work")).unwrap(); + assert!( + f.cli_calls.borrow().is_empty(), + "static keys are right there; spawning the AWS CLI would be silly" + ); + } + + #[test] + fn an_unknown_profile_says_so_rather_than_shelling_out() { + let f = Fake::new().credentials(STATIC_PROFILE); + let err = f.resolve(None, Some("nope")).unwrap_err().to_string(); + assert!(err.contains("nope"), "{err}"); + assert!(f.cli_calls.borrow().is_empty()); + } + + #[test] + fn region_comes_from_the_profile_when_we_have_none() { + // `~/.aws/config` spells non-default profiles `[profile name]`. + let f = Fake::new() + .credentials(STATIC_PROFILE) + .config("[default]\nregion = us-east-2\n\n[profile work]\nregion = eu-west-1\n"); + assert_eq!( + f.resolve(None, Some("work")).unwrap().region.as_deref(), + Some("eu-west-1") + ); + assert_eq!( + f.resolve(None, None).unwrap().region.as_deref(), + Some("us-east-2") + ); + } + + #[test] + fn credentials_file_location_honors_the_aws_env_overrides() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("elsewhere"); + std::fs::write(&path, STATIC_PROFILE).unwrap(); + let f = Fake::new().var("AWS_SHARED_CREDENTIALS_FILE", path.to_str().unwrap()); + assert_eq!( + f.resolve(None, None) + .unwrap() + .credentials + .unwrap() + .access_key_id, + "AKIADEFAULT" + ); + } + + // ── expired SSO ────────────────────────────────────────────────── + + /// What the AWS CLI actually prints when an SSO session lapses. + const EXPIRED: &str = "the AWS CLI could not resolve this profile: \ +Error loading SSO Token: Token for https://corp.awsapps.com/start does not exist"; + + const FRESH: &str = + r#"{"Version":1,"AccessKeyId":"ASIAFRESH","SecretAccessKey":"s","SessionToken":"t"}"#; + + fn sso_profile() -> Fake { + Fake::new().config("[profile sso-work]\nsso_session = corp\n") + } + + #[test] + fn an_expired_sso_session_offers_to_log_in_and_retries_once() { + let f = sso_profile().cli_err(EXPIRED).cli_after_login(FRESH); + let r = f.resolve(None, Some("sso-work")).unwrap(); + + assert_eq!(r.credentials.unwrap().access_key_id, "ASIAFRESH"); + assert_eq!(*f.login_calls.borrow(), vec!["sso-work".to_string()]); + // Asked before opening a browser, and named the command. + let q = f.questions.borrow().join(""); + assert!(q.contains("aws sso login --profile sso-work"), "{q}"); + // One retry, not a loop. + assert_eq!(f.cli_calls.borrow().len(), 2); + } + + #[test] + fn declining_the_offer_prints_the_command_and_logs_in_to_nothing() { + let f = sso_profile().cli_err(EXPIRED).declines(); + let err = format!("{:#}", f.resolve(None, Some("sso-work")).unwrap_err()); + + assert!(err.contains("aws sso login --profile sso-work"), "{err}"); + assert!( + f.login_calls.borrow().is_empty(), + "declining must not open a browser" + ); + } + + #[test] + fn a_failed_login_surfaces_rather_than_retrying_blindly() { + let f = sso_profile().cli_err(EXPIRED).login_fails(); + let err = format!("{:#}", f.resolve(None, Some("sso-work")).unwrap_err()); + assert!(err.contains("did not complete"), "{err}"); + assert_eq!( + f.cli_calls.borrow().len(), + 1, + "no retry after a failed login" + ); + } + + #[test] + fn a_login_that_still_yields_nothing_reports_the_second_failure() { + // Logged in, still broken: the real error is whatever comes back + // now, and looping wouldn't help. + let f = sso_profile().cli_err(EXPIRED); + let err = format!("{:#}", f.resolve(None, Some("sso-work")).unwrap_err()); + assert!(err.contains("after `aws sso login`"), "{err}"); + assert_eq!(f.cli_calls.borrow().len(), 2); + } + + #[test] + fn an_unrelated_cli_failure_never_offers_a_login() { + // Only an expired *SSO session* has this obvious next step. + let f = sso_profile().cli_err( + "the AWS CLI could not resolve this profile: \ +Unable to locate credentials for role arn:aws:iam::1:role/nope", + ); + let err = format!("{:#}", f.resolve(None, Some("sso-work")).unwrap_err()); + assert!(err.contains("Unable to locate credentials"), "{err}"); + assert!(f.login_calls.borrow().is_empty()); + assert!(f.questions.borrow().is_empty(), "nothing to ask about"); + } + + #[test] + fn a_working_sso_profile_is_not_asked_about() { + let f = sso_profile().cli(FRESH); + f.resolve(None, Some("sso-work")).unwrap(); + assert!(f.questions.borrow().is_empty()); + assert!(f.login_calls.borrow().is_empty()); + } + + #[test] + fn expired_sso_detection_covers_the_wordings_the_cli_uses() { + for msg in [ + "Error loading SSO Token: Token for https://x/start does not exist", + "The SSO session associated with this profile has expired", + "Error when retrieving token from sso: Token has expired and refresh failed", + "To refresh this SSO session run aws sso login with the corresponding profile", + ] { + assert!(is_expired_sso(&anyhow::anyhow!("{msg}")), "missed: {msg}"); + } + // And doesn't fire on failures with a different fix. + for msg in [ + "Unable to locate credentials", + "An error occurred (AccessDenied) when calling AssumeRole", + "Could not connect to the endpoint URL", + ] { + assert!( + !is_expired_sso(&anyhow::anyhow!("{msg}")), + "false positive: {msg}" + ); + } + } + + // ── ini parsing ────────────────────────────────────────────────── + + #[test] + fn ini_ignores_comments_and_nested_subsections() { + let ini = parse_ini( + "\ +# a comment +; another +[default] +aws_access_key_id = AKIA ; trailing content is part of the value +region=us-east-1 + +[services x] + s3 = + endpoint_url = http://nested +", + ); + assert_eq!(ini["default"]["region"], "us-east-1"); + // Indented sub-section bodies must not leak into the section. + assert!(!ini["services x"].contains_key("endpoint_url")); + } + + #[test] + fn ini_keys_are_case_insensitive() { + let ini = parse_ini("[default]\nAWS_ACCESS_KEY_ID = AKIA\n"); + assert_eq!(ini["default"]["aws_access_key_id"], "AKIA"); + } + + #[test] + fn export_credentials_output_without_keys_is_an_error() { + let err = parse_export_credentials(r#"{"Version":1}"#) + .unwrap_err() + .to_string(); + assert!(err.contains("no credentials"), "{err}"); + } + + #[test] + fn export_credentials_non_json_is_an_error() { + let err = parse_export_credentials("Unable to locate credentials") + .unwrap_err() + .to_string(); + assert!(err.contains("isn't JSON"), "{err}"); + } +} diff --git a/crates/path-cli/src/cmd_auth.rs b/crates/path-cli/src/cmd_auth.rs index 621dbc4a..3dcbb6bf 100644 --- a/crates/path-cli/src/cmd_auth.rs +++ b/crates/path-cli/src/cmd_auth.rs @@ -1,4 +1,8 @@ use anyhow::{Result, anyhow}; +use clap::Args; +use std::io::IsTerminal; + +use crate::store::{self, S3Settings}; use clap::Subcommand; use std::path::Path; @@ -25,15 +29,86 @@ pub enum AuthOp { Status, /// Verify the stored session against the server and print the current user Whoami, + /// Store S3 credentials for endpoints AWS tooling doesn't know + /// about (MinIO, R2, Ceph). Your `~/.aws` profiles — SSO included — + /// are picked up automatically and need none of this. + S3 { + #[command(subcommand)] + op: S3Op, + }, +} + +#[derive(Subcommand, Debug)] +pub enum S3Op { + /// Store S3 credentials and connection settings. + /// + /// Only the fields you pass are updated; the rest keep their stored + /// values, so `path auth s3 login --region eu-west-1` is a valid + /// tweak. Run interactively with no flags and it prompts, without + /// echoing the secret. + /// + /// This does not set *where* shares go — that's `path target` + /// — so one stored credential serves any number of buckets. + #[command(alias = "set")] + Login { + #[command(flatten)] + args: S3LoginArgs, + }, + /// Show the S3 settings in effect, with secrets redacted and + /// environment-supplied values marked + Status, + /// Forget the stored S3 settings + #[command(alias = "clear")] + Logout, +} + +#[derive(Args, Debug, Default)] +pub struct S3LoginArgs { + /// AWS region (default: us-east-1) + #[arg(long)] + pub region: Option, + + /// Endpoint URL for an S3-compatible service (R2, MinIO, Ceph). + /// Omit for real AWS S3. + #[arg(long)] + pub endpoint: Option, + + #[arg(long)] + pub access_key_id: Option, + + /// Secret access key. Prefer omitting this so it's prompted for + /// rather than landing in your shell history. + #[arg(long)] + pub secret_access_key: Option, + + /// Session token for temporary (STS / assumed-role) credentials + #[arg(long)] + pub session_token: Option, + + /// AWS profile to resolve credentials from, instead of storing keys. + /// Works with SSO and assume-role profiles — those are resolved + /// through the AWS CLI, so nothing expires in our config. + #[arg(long)] + pub profile: Option, + + /// Address the bucket as `bucket.host/key` instead of `host/bucket/key` + #[arg(long)] + pub virtual_hosted_style: bool, } pub fn run(op: AuthOp) -> Result<()> { - let path = credentials_path()?; match op { - AuthOp::Login { url, code } => login(&path, url, code), - AuthOp::Logout => logout(&path), - AuthOp::Status => status(&path), - AuthOp::Whoami => whoami(&path), + AuthOp::S3 { op } => run_s3(op), + other => { + let path = credentials_path()?; + match other { + AuthOp::Login { url, code } => login(&path, url, code), + AuthOp::Logout => logout(&path), + AuthOp::Status => status(&path), + AuthOp::Whoami => whoami(&path), + AuthOp::S3 { .. } => unreachable!("handled above"), + } + } } } @@ -113,6 +188,202 @@ fn status(path: &Path) -> Result<()> { } } +// ── S3 ────────────────────────────────────────────────────────────────── + +fn run_s3(op: S3Op) -> Result<()> { + let path = store::config_path()?; + match op { + S3Op::Login { args } => s3_login(&path, args), + S3Op::Status => s3_status(&path), + S3Op::Logout => s3_logout(&path), + } +} + +/// Merge `args` into whatever is already stored, prompting for the +/// essentials when nothing was passed and we have a terminal. +/// +/// Merge rather than replace: partial updates are the common case +/// (rotating a key, switching endpoint), and a replace would silently +/// drop the fields the user didn't repeat. +fn s3_login(path: &Path, args: S3LoginArgs) -> Result<()> { + let mut cfg = store::load_stored(path)?.unwrap_or_default(); + let had_settings = cfg != S3Settings::default(); + + let set = |slot: &mut Option, value: Option| { + if let Some(v) = value + .map(|v| v.trim().to_string()) + .filter(|v| !v.is_empty()) + { + *slot = Some(v); + } + }; + set(&mut cfg.region, args.region); + set(&mut cfg.endpoint, args.endpoint); + set(&mut cfg.access_key_id, args.access_key_id); + set(&mut cfg.secret_access_key, args.secret_access_key); + set(&mut cfg.session_token, args.session_token); + set(&mut cfg.profile, args.profile); + if args.virtual_hosted_style { + cfg.virtual_hosted_style = Some(true); + } + + if std::io::stdin().is_terminal() && !had_settings { + prompt_missing(&mut cfg)?; + } + + if cfg == S3Settings::default() { + anyhow::bail!( + "Nothing to store. Pass at least one setting (e.g. \ + `path auth s3 login --access-key-id AKIA… --secret-access-key …`), \ + or run this from a terminal to be prompted." + ); + } + + store::store(path, &cfg)?; + println!("S3 settings saved to {}", path.display()); + // Everything printed here was just written, so nothing is `(env)`. + print_settings(&cfg, &cfg); + Ok(()) +} + +/// First-time interactive setup. Skipped when settings already exist, +/// so a targeted `--region` update doesn't re-interrogate the user +/// about credentials they already stored. +fn prompt_missing(cfg: &mut S3Settings) -> Result<()> { + println!("Store S3 connection settings for `s3://` share and resume targets."); + println!(); + println!("If you already use the AWS CLI, you probably need none of this — your"); + println!("`~/.aws` profiles are picked up automatically, including SSO. This is"); + println!("for endpoints the AWS tooling doesn't know about (MinIO, R2, Ceph)."); + println!("Leave any field blank to skip it."); + println!(); + + if cfg.region.is_none() { + let v = prompt_line(&format!("Region [{}]: ", store::DEFAULT_REGION))?; + cfg.region = Some(v).filter(|v| !v.is_empty()); + } + if cfg.endpoint.is_none() { + let v = prompt_line("Endpoint URL (blank for AWS): ")?; + cfg.endpoint = Some(v).filter(|v| !v.is_empty()); + } + if cfg.access_key_id.is_none() { + let v = prompt_line("Access key id (blank to use the AWS environment): ")?; + cfg.access_key_id = Some(v).filter(|v| !v.is_empty()); + } + if cfg.access_key_id.is_some() && cfg.secret_access_key.is_none() { + let v = rpassword::prompt_password("Secret access key: ")?; + cfg.secret_access_key = Some(v.trim().to_string()).filter(|v| !v.is_empty()); + } + Ok(()) +} + +fn s3_status(path: &Path) -> Result<()> { + let stored = store::load_stored(path)?; + let effective = store::effective_settings()?; + + match &stored { + Some(_) => println!("S3 settings in {}", path.display()), + None => println!("No stored S3 settings ({} does not exist).", path.display()), + } + if effective == S3Settings::default() { + println!("Run `path auth s3 login` to store some."); + } else { + print_settings(&effective, &stored.unwrap_or_default()); + } + print_credential_source(&effective); + Ok(()) +} + +fn s3_logout(path: &Path) -> Result<()> { + if store::load_stored(path)?.is_none() { + println!("No stored S3 settings."); + return Ok(()); + } + store::clear(path)?; + println!("S3 settings cleared."); + Ok(()) +} + +/// Print `effective`, tagging any field that `stored` didn't supply as +/// `(env)` — otherwise "where did this endpoint come from?" is a guess. +fn print_settings(effective: &S3Settings, stored: &S3Settings) { + let line = |label: &str, value: Option<&str>, from_store: bool| { + if let Some(v) = value { + let origin = if from_store { "" } else { " (env)" }; + // Width matches the longest label so the values line up. + println!(" {:<19}{v}{origin}", format!("{label}:")); + } + }; + line( + "region", + effective.region.as_deref(), + stored.region.is_some(), + ); + line( + "endpoint", + effective.endpoint.as_deref(), + stored.endpoint.is_some(), + ); + line( + "access key id", + effective.access_key_id.as_deref(), + stored.access_key_id.is_some(), + ); + line( + "secret access key", + effective + .secret_access_key + .as_deref() + .map(redact) + .as_deref(), + stored.secret_access_key.is_some(), + ); + line( + "session token", + effective.session_token.as_deref().map(redact).as_deref(), + stored.session_token.is_some(), + ); + line( + "profile", + effective.profile.as_deref(), + stored.profile.is_some(), + ); +} + +/// Say which credentials a share would actually use. +/// +/// The first question when an upload fails is *which* credential was +/// tried — a stored key, an AWS profile, or nothing at all are three +/// completely different fixes, and only this line distinguishes them. +fn print_credential_source(effective: &S3Settings) { + match effective.resolve_real() { + Ok(r) => { + println!(" credentials: {}", r.source); + if let Some(region) = &r.region + && effective.region.is_none() + { + println!(" region: {region} (from the profile)"); + } + } + // The reason *is* the answer here — "no such profile" tells the + // user exactly what to fix. + Err(e) => println!(" credentials: unresolved — {e:#}"), + } +} + +/// Show enough of a secret to recognize which one it is, and no more. +fn redact(secret: &str) -> String { + let tail: String = secret + .chars() + .rev() + .take(4) + .collect::>() + .into_iter() + .rev() + .collect(); + format!("****{tail}") +} + fn whoami(path: &Path) -> Result<()> { let stored = load_session(path)?.ok_or_else(|| anyhow!("Not logged in. Run `path auth login`."))?; diff --git a/crates/path-cli/src/cmd_export.rs b/crates/path-cli/src/cmd_export.rs index b95a67f0..8a3df1ab 100644 --- a/crates/path-cli/src/cmd_export.rs +++ b/crates/path-cli/src/cmd_export.rs @@ -200,6 +200,24 @@ pub enum ExportTarget { #[arg(long)] public: bool, }, + /// Upload a toolpath document to object storage: an S3 bucket, an + /// S3-compatible endpoint, or a plain folder. + /// + /// S3 credentials come from your `~/.aws` profiles, the AWS + /// environment, or `path auth s3 login`; a folder needs none. The + /// object is named `--.json`, and the + /// printed location is what `path resume` takes. + #[command(alias = "s3")] + Object { + /// Input: cache id (e.g. `claude-abc`) or path to a toolpath JSON file + #[arg(short, long)] + input: String, + + /// Destination: `s3://bucket/prefix`, or a folder (`~/traces`, + /// `file:///srv/traces`). + #[arg(long, value_name = "DESTINATION")] + to: String, + }, } pub fn run(target: ExportTarget) -> Result<()> { @@ -255,6 +273,7 @@ pub fn run(target: ExportTarget) -> Result<()> { name, public, }), + ExportTarget::Object { input, to } => run_object(input, to), } } @@ -1895,6 +1914,41 @@ fn write_cursor_to_stdout(session: &toolpath_cursor::CursorSession) -> Result<() // ── Pathbase ────────────────────────────────────────────────────────── +// ── Object storage ──────────────────────────────────────────────────── + +fn run_object(input: String, to: String) -> Result<()> { + #[cfg(target_os = "emscripten")] + { + let _ = (input, to); + anyhow::bail!("'path p export object' requires a native environment with network access"); + } + + #[cfg(not(target_os = "emscripten"))] + { + let file = cache_ref(&input)?; + let body = std::fs::read_to_string(&file) + .with_context(|| format!("Failed to read {}", file.display()))?; + + let dest = crate::store::Destination::parse(&to)?; + let settings = crate::store::effective_settings()?; + + // The object is named for the cache id, not the input string — + // `--input ./some/file.json` and `--input claude-abc` naming the + // same document should land on the same key. + let cache_id = file + .file_stem() + .map(|s| s.to_string_lossy().into_owned()) + .unwrap_or_else(|| input.clone()); + + let uri = dest.uri_for(&crate::store::name_for_body(&body, &cache_id)); + uri.put(&settings, body.as_bytes())?; + println!("{uri}"); + eprintln!("Uploaded {} bytes → {uri}", body.len()); + eprintln!("Resume it with: path resume {uri}"); + Ok(()) + } +} + fn run_pathbase(args: PathbaseExportArgs) -> Result<()> { #[cfg(target_os = "emscripten")] { diff --git a/crates/path-cli/src/cmd_import.rs b/crates/path-cli/src/cmd_import.rs index 7d210beb..6edc355c 100644 --- a/crates/path-cli/src/cmd_import.rs +++ b/crates/path-cli/src/cmd_import.rs @@ -183,6 +183,17 @@ pub enum ImportSource { #[arg(long)] url: Option, }, + /// Import from object storage — an S3 bucket, an S3-compatible + /// endpoint, or a folder. S3 credentials come from your `~/.aws` + /// profiles, the AWS environment, or `path auth s3 login`; a folder + /// needs none. + #[command(alias = "s3")] + Object { + /// Object URL: `s3://bucket/key.json` (also `s3a://`, and + /// `file:///dir/key.json` for a local folder) + #[arg(index = 1)] + target: String, + }, } #[derive(clap::Args, Debug)] @@ -301,6 +312,7 @@ fn derive(source: ImportSource) -> Result> { base, } => derive_pi(project, session, all, base), ImportSource::Pathbase { target, url } => derive_pathbase(target, url), + ImportSource::Object { target } => derive_object(target), } } @@ -1531,6 +1543,19 @@ fn derive_pathbase(target: String, url_flag: Option) -> Result Result> { + #[cfg(target_os = "emscripten")] + { + let _ = target; + anyhow::bail!("'path p import object' requires a native environment with network access"); + } + + #[cfg(not(target_os = "emscripten"))] + { + Ok(vec![crate::derive::object_fetch_to_doc(&target)?]) + } +} + #[cfg(all(test, not(target_os = "emscripten")))] mod tests { use super::*; diff --git a/crates/path-cli/src/cmd_pathbase.rs b/crates/path-cli/src/cmd_pathbase.rs index bfff3f7b..5dda38be 100644 --- a/crates/path-cli/src/cmd_pathbase.rs +++ b/crates/path-cli/src/cmd_pathbase.rs @@ -332,7 +332,10 @@ fn short_body(body: &str) -> String { // hand-rolled only because the redeem endpoint isn't in the OpenAPI // spec, not because of any HTTP-stack difference. -fn block_on(f: F) -> F::Output { +/// Shared with the `store` module, which drives the equally-async +/// `object_store` client: one runtime for every async client the CLI +/// tunnels into. +pub(crate) fn block_on(f: F) -> F::Output { use std::sync::OnceLock; static RT: OnceLock = OnceLock::new(); let rt = RT.get_or_init(|| { diff --git a/crates/path-cli/src/cmd_resume.rs b/crates/path-cli/src/cmd_resume.rs index 4b0b6e3d..d5b9290d 100644 --- a/crates/path-cli/src/cmd_resume.rs +++ b/crates/path-cli/src/cmd_resume.rs @@ -193,6 +193,131 @@ pub(crate) fn ensure_path_with_agent(g: &Graph) -> Result<&TPath> { Ok(path) } +/// Fetch an object, preferring an existing cache entry. +/// +/// `--force` skips the probe and re-fetches; `--no-cache` skips both +/// the probe AND the post-fetch write. The cache id comes from the URI +/// alone, which is what makes a cache hit cost no round trip. +fn fetch_object_cached(label: &str, cache_id: &str, args: &ResumeArgs, uri: &str) -> Result { + if !args.force + && !args.no_cache + && let Ok(cache_path) = crate::cache::cache_path(cache_id) + && cache_path.exists() + { + let json = std::fs::read_to_string(&cache_path) + .with_context(|| format!("read {}", cache_path.display()))?; + eprintln!("Resolved {label} → {cache_id} (cached)"); + return Graph::from_json(&json) + .map_err(|e| anyhow::anyhow!("cached toolpath document is invalid: {}", e)); + } + + let derived = crate::derive::object_fetch_to_doc(uri)?; + if !args.no_cache { + crate::cache::write_cached(&derived.cache_id, &derived.doc, true)?; + eprintln!("Resolved {label} → {}", derived.cache_id); + } + Ok(derived.doc) +} + +/// List a share destination and let the user pick one of its documents. +/// +/// This is the other half of sharing to object storage: without it a +/// destination is a write-only hole you can only read out of by already +/// knowing a filename. Rows are built from object names alone — no +/// downloads — which is what legible names buy. +fn pick_from_destination(raw: &str) -> Result { + let dest = crate::store::Destination::parse(raw)?; + let settings = crate::store::effective_settings()?; + let entries = dest.list(&settings)?; + + if entries.is_empty() { + anyhow::bail!( + "no shared documents in {dest}. `path p export object --to {dest}` puts one there." + ); + } + + // One document is not a choice worth making the user confirm. + if entries.len() == 1 { + let only = &entries[0]; + eprintln!("Resuming the only document in {dest}: {}", only.stem); + return Ok(only.uri.clone()); + } + + if !crate::fuzzy::available() { + eprintln!("{} documents in {dest}:", entries.len()); + for e in entries.iter().take(20) { + eprintln!(" {}", e.uri); + } + if entries.len() > 20 { + eprintln!(" … and {} more", entries.len() - 20); + } + anyhow::bail!("picking needs `fzf` on PATH and a TTY; pass a full location instead"); + } + + let lines: Vec = entries + .iter() + .enumerate() + .map(|(i, e)| format_object_row(i, e)) + .collect(); + let header = format!("resume a shared session from {dest}"); + let opts = crate::fuzzy::PickOptions { + with_nth: "2..", + prompt: "resume> ", + preview: None, + preview_window: "up:60%:wrap-word", + header: Some(&header), + tiebreak: "index", + multi: false, + }; + let line = match crate::fuzzy::pick(&lines, &opts)? { + crate::fuzzy::PickResult::Selected(v) => match v.into_iter().next() { + Some(l) => l, + None => std::process::exit(130), + }, + crate::fuzzy::PickResult::NoMatch => std::process::exit(1), + // Esc / Ctrl-C: a deliberate cancel. + crate::fuzzy::PickResult::Cancelled => std::process::exit(130), + }; + let idx: usize = line + .split('\t') + .next() + .and_then(|i| i.parse().ok()) + .ok_or_else(|| anyhow::anyhow!("internal: failed to parse picker row"))?; + entries + .get(idx) + .map(|e| e.uri.clone()) + .ok_or_else(|| anyhow::anyhow!("internal: picker returned an out-of-range row")) +} + +/// `\t ` — a hidden index column so the +/// selection maps back to its entry without reparsing the display. +fn format_object_row(index: usize, entry: &crate::store::ObjectEntry) -> String { + let when = entry + .modified + .map(|t| t.format("%Y-%m-%d").to_string()) + .unwrap_or_else(|| " ".to_string()); + format!( + "{index}\t{when} {:>8} {}", + human_size(entry.size), + entry.stem + ) +} + +fn human_size(bytes: u64) -> String { + const UNITS: [&str; 4] = ["B", "KB", "MB", "GB"]; + let mut size = bytes as f64; + let mut unit = 0; + while size >= 1024.0 && unit + 1 < UNITS.len() { + size /= 1024.0; + unit += 1; + } + if unit == 0 { + format!("{bytes} B") + } else { + format!("{size:.1} {}", UNITS[unit]) + } +} + /// Resolve the user-supplied `` argument into a parsed `Graph` /// plus the source harness inferred from its single inline path (if /// any). See spec § "Input resolution" for the order. @@ -202,14 +327,31 @@ pub(crate) fn resolve_input(args: &ResumeArgs) -> Result<(Graph, Option enum Shape<'a> { PathbaseUrl(&'a str), PathbaseShorthand(&'a str), + ObjectStore(&'a str), + /// A share destination rather than a single document: list it + /// and let the user pick. + ObjectContainer(&'a str), FilePath(&'a str), CacheId(&'a str), } + // A shared document is always `.json`, so anything else that + // names a place — a bucket, a prefix, a directory — is a container + // to browse rather than a document to load. + let names_a_document = raw.trim_end_matches('/').ends_with(".json"); + let shape = if raw.starts_with("http://") || raw.starts_with("https://") { Shape::PathbaseUrl(raw) + } else if crate::store::looks_like_object_uri(raw) { + if names_a_document { + Shape::ObjectStore(raw) + } else { + Shape::ObjectContainer(raw) + } } else if looks_like_pathbase_shorthand(raw) { Shape::PathbaseShorthand(raw) + } else if std::path::Path::new(raw).is_dir() { + Shape::ObjectContainer(raw) } else if std::path::Path::new(raw).is_file() { Shape::FilePath(raw) } else { @@ -247,6 +389,17 @@ pub(crate) fn resolve_input(args: &ResumeArgs) -> Result<(Graph, Option derived.doc } } + Shape::ObjectStore(u) => { + // Same shape as Pathbase: the cache id falls out of the URI + // alone, so an already-downloaded object costs no request. + let cache_id = crate::store::ObjectUri::parse(u)?.cache_id(); + fetch_object_cached(raw, &cache_id, args, u)? + } + Shape::ObjectContainer(c) => { + let picked = pick_from_destination(c)?; + let uri = picked.to_string(); + fetch_object_cached(&uri, &picked.cache_id(), args, &uri)? + } Shape::FilePath(p) => { let json = std::fs::read_to_string(p).with_context(|| format!("read {}", p))?; Graph::from_json(&json) diff --git a/crates/path-cli/src/config.rs b/crates/path-cli/src/config.rs index 9a76e2cb..2467defd 100644 --- a/crates/path-cli/src/config.rs +++ b/crates/path-cli/src/config.rs @@ -39,6 +39,9 @@ pub(crate) const MANIFEST_LOCK_FILE_NAME: &str = "manifest.json.lock"; pub(crate) const CREDENTIALS_FILE_NAME: &str = "credentials.json"; /// The document cache directory (see `cache`). pub(crate) const DOCUMENTS_DIR_NAME: &str = "documents"; +/// S3 connection settings, written by `path auth s3 login` +/// (see `store`). +pub(crate) const S3_SETTINGS_FILE_NAME: &str = "s3.json"; /// Environment-derived configuration. [`Config::load`] reads the /// environment once, at the composition root. Code below the root @@ -179,6 +182,52 @@ pub(crate) fn home_relative(path: &std::path::Path, home: Option<&std::path::Pat path.display().to_string() } +/// Write `value` as pretty JSON to a user-private file: the parent +/// directory is created `0700` and the file itself `0600`. +/// +/// Every credential-bearing blob under the config dir goes through here +/// (Pathbase sessions, S3 settings) so the permissions story is stated +/// once instead of re-derived per call site. +pub(crate) fn write_private_json( + path: &std::path::Path, + value: &T, +) -> Result<()> { + let parent = path + .parent() + .ok_or_else(|| anyhow!("config path has no parent: {}", path.display()))?; + std::fs::create_dir_all(parent).map_err(|e| anyhow!("create {}: {e}", parent.display()))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let _ = std::fs::set_permissions(parent, std::fs::Permissions::from_mode(0o700)); + } + + let payload = serde_json::to_string_pretty(value)?; + std::fs::write(path, payload).map_err(|e| anyhow!("write {}: {e}", path.display()))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)) + .map_err(|e| anyhow!("chmod 0600 {}: {e}", path.display()))?; + } + Ok(()) +} + +/// Read a JSON blob written by [`write_private_json`]. A missing or +/// empty file is `Ok(None)` — "not configured", not an error. +pub(crate) fn read_private_json( + path: &std::path::Path, +) -> Result> { + match std::fs::read_to_string(path) { + Ok(s) if s.trim().is_empty() => Ok(None), + Ok(s) => Ok(Some( + serde_json::from_str(&s).map_err(|e| anyhow!("decode {}: {e}", path.display()))?, + )), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None), + Err(e) => Err(anyhow!("read {}: {e}", path.display())), + } +} + /// Shared lock for tests that manipulate `$TOOLPATH_CONFIG_DIR`. Every /// test module that calls `set_var` / `remove_var` on this env var should /// grab this lock first, otherwise parallel tests race and clobber each diff --git a/crates/path-cli/src/derive.rs b/crates/path-cli/src/derive.rs index d9b09e89..08e02090 100644 --- a/crates/path-cli/src/derive.rs +++ b/crates/path-cli/src/derive.rs @@ -352,6 +352,27 @@ pub(crate) fn derive_pi_session_with( }) } +/// Download a toolpath document from object storage (`s3://bucket/key`, +/// `file:///dir/doc.json`) and parse it. The counterpart to +/// `p export object`; used by `p import object` and `path resume`. +/// +/// `provenance` is `None` for the same reason Pathbase downloads leave +/// it unset: the sync manifest tracks local artifact sources, and a +/// remote object isn't one. +#[cfg(not(target_os = "emscripten"))] +pub(crate) fn object_fetch_to_doc(target: &str) -> Result { + let uri = crate::store::ObjectUri::parse(target)?; + let cfg = crate::store::effective_settings()?; + let body = uri.get(&cfg)?; + let doc = Graph::from_json(&body) + .map_err(|e| anyhow::anyhow!("{uri} is not a toolpath document: {e}"))?; + Ok(DerivedDoc { + cache_id: uri.cache_id(), + doc, + provenance: None, + }) +} + /// Fetch a Pathbase ref (`https://host/u/owner/repos/repo/graphs/` /// URL or bare `owner/repo/` triple) and parse it as a toolpath /// document. Used by `path import pathbase` and `path resume `. diff --git a/crates/path-cli/src/lib.rs b/crates/path-cli/src/lib.rs index 43cbbd29..00fef1d8 100644 --- a/crates/path-cli/src/lib.rs +++ b/crates/path-cli/src/lib.rs @@ -1,4 +1,6 @@ pub mod artifact; +#[cfg(not(target_os = "emscripten"))] +mod aws_creds; mod cache; #[cfg(not(target_os = "emscripten"))] mod cmd_auth; @@ -44,6 +46,8 @@ mod schema; mod share_config; #[cfg(all(not(target_os = "emscripten"), feature = "embedded-picker"))] mod skim_picker; +#[cfg(not(target_os = "emscripten"))] +mod store; mod sync; mod term; diff --git a/crates/path-cli/src/store.rs b/crates/path-cli/src/store.rs new file mode 100644 index 00000000..e4f8e4d8 --- /dev/null +++ b/crates/path-cli/src/store.rs @@ -0,0 +1,1141 @@ +//! Object-storage destinations for share and resume. +//! +//! Transport is [`object_store`], so one code path covers real AWS S3, +//! any S3-compatible endpoint (Cloudflare R2, MinIO, Ceph, Backblaze +//! B2), and a plain local directory via `file://`. A folder is a +//! first-class destination, not a testing affordance: `path target +//! ~/Dropbox/traces` is a complete setup, needing no credentials at +//! all. It is also what the tests round-trip against, so share and +//! resume are exercised end-to-end without a network. +//! +//! The module owns two separable things: +//! +//! 1. [`Destination`] / [`ObjectUri`] / [`ObjectName`] — *where* a +//! document goes and what it's called. Pure URL parsing and naming; +//! no credentials involved. Where a document lands is a function of +//! the destination and the document itself, nothing else. +//! 2. [`S3Settings`] — *how to reach* an `s3://` destination: region, +//! endpoint, addressing style, and credentials, persisted at +//! `~/.toolpath/s3.json` by `path auth s3 login`. +//! +//! Keeping those apart is what lets `--to ~/traces` skip the whole +//! credential story, and lets one stored credential serve any number +//! of buckets. +//! +//! Credentials are handed to `object_store` as config options rather +//! than resolved here. When none are configured, the AWS credential +//! chain (env, EC2/ECS instance metadata, web identity) still applies — +//! so an EC2 box with an instance role needs no `path auth s3 login` +//! at all. + +use anyhow::{Context, Result, anyhow, bail}; +use object_store::{ObjectStore, ObjectStoreExt}; +use serde::{Deserialize, Serialize}; +use std::path::PathBuf; +use url::Url; + +use crate::config::config_dir; + +pub(crate) const DEFAULT_REGION: &str = "us-east-1"; + +/// URL schemes routed to object storage. Deliberately narrower than +/// what `object_store` can parse: `http`/`https` belong to Pathbase in +/// every command that shares this dispatch, `gs://` / `az://` would +/// need feature flags we don't compile in, and `memory://` is a fresh +/// per-process store — anything "shared" there is gone before the +/// command exits, so accepting it would only waste someone's afternoon. +const SCHEMES: [&str; 3] = ["s3", "s3a", "file"]; + +// ── S3 connection settings ────────────────────────────────────────────── + +/// The blob persisted at `~/.toolpath/s3.json` (0600). +/// +/// Connection and credentials only — deliberately *not* a destination. +/// A destination is named per call (`--to s3://bucket/prefix`), so one +/// stored credential serves any number of buckets, and a folder +/// destination needs no credentials at all. +/// +/// Every field is optional so a partial configuration is legal: a user +/// whose credentials come from the environment (CI, an EC2 instance +/// role) may store only `region`/`endpoint`, or nothing at all. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub(crate) struct S3Settings { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub region: Option, + /// Custom endpoint (`https://…`) for S3-compatible services such as + /// Cloudflare R2 or MinIO. Absent means real AWS S3. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub endpoint: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub access_key_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub secret_access_key: Option, + /// Temporary-credential token (STS / assumed role). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub session_token: Option, + /// Use virtual-hosted addressing (`bucket.host/key`) instead of + /// path style (`host/bucket/key`). Unset lets `object_store` pick: + /// path style, which every S3-compatible endpoint accepts. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub virtual_hosted_style: Option, + /// AWS profile to resolve credentials from, when you don't want the + /// `AWS_PROFILE` / `[default]` answer. Storing a profile name is + /// very different from storing a key: it's a pointer to credentials + /// the AWS tooling already manages, so it can't go stale and it + /// costs nothing at rest. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub profile: Option, +} + +/// The credential resolution this settings blob implies. +/// +/// `profile` is threaded through so `--profile` on a command reaches +/// the resolver; everything else comes from the ambient AWS setup. +impl S3Settings { + pub(crate) fn resolved_credentials(&self) -> Option { + self.resolve_real().ok() + } + + /// [`resolved_credentials`](Self::resolved_credentials), keeping the + /// error. Anything *reporting* on credentials wants the reason — + /// "no such profile" is the whole answer, and swallowing it leaves + /// the user with nothing to act on. + pub(crate) fn resolve_real(&self) -> Result { + self.resolve_with(&crate::aws_creds::Env { + home: std::env::var_os("HOME").map(PathBuf::from), + var: &|k| std::env::var(k).ok(), + aws_cli: &crate::aws_creds::run_aws_cli, + sso_login: &crate::aws_creds::run_sso_login, + confirm: &crate::aws_creds::confirm_on_tty, + }) + } + + /// [`resolved_credentials`](Self::resolved_credentials) against an + /// injected environment, and propagating the error so callers that + /// want to *report* a failure (rather than fall through to the + /// instance chain) can. + pub(crate) fn resolve_with( + &self, + env: &crate::aws_creds::Env<'_>, + ) -> Result { + let stored = match (&self.access_key_id, &self.secret_access_key) { + (Some(k), Some(s)) => Some(crate::aws_creds::Credentials { + access_key_id: k.clone(), + secret_access_key: s.clone(), + session_token: self.session_token.clone(), + }), + _ => None, + }; + crate::aws_creds::resolve(stored, self.profile.as_deref(), env) + } +} + +pub(crate) fn config_path() -> Result { + Ok(config_dir()?.join(crate::config::S3_SETTINGS_FILE_NAME)) +} + +pub(crate) fn load_stored(path: &std::path::Path) -> Result> { + crate::config::read_private_json(path) +} + +pub(crate) fn store(path: &std::path::Path, cfg: &S3Settings) -> Result<()> { + crate::config::write_private_json(path, cfg) +} + +pub(crate) fn clear(path: &std::path::Path) -> Result<()> { + match std::fs::remove_file(path) { + Ok(()) => Ok(()), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(e) => Err(anyhow!("remove {}: {e}", path.display())), + } +} + +/// The stored settings with environment variables filling any gap. +/// +/// Precedence is stored-then-env, not env-then-stored: the point of +/// `path auth s3 login` is that what you configured is what you get. +/// Env vars are the fallback for environments that never ran `login` +/// (CI, containers), and they use the conventional AWS names so an +/// already-configured shell just works. +/// +/// Recognized: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, +/// `AWS_SESSION_TOKEN`, `AWS_REGION` (then `AWS_DEFAULT_REGION`), and +/// `AWS_ENDPOINT_URL_S3` (then `AWS_ENDPOINT_URL`). +pub(crate) fn effective_settings() -> Result { + let stored = load_stored(&config_path()?)?.unwrap_or_default(); + Ok(merge_env(stored, |k| std::env::var(k).ok())) +} + +/// [`effective_settings`] with the environment injected, so tests don't +/// have to mutate process-global state. +pub(crate) fn merge_env Option>(mut cfg: S3Settings, env: F) -> S3Settings { + let first = |keys: &[&str]| -> Option { + keys.iter() + .find_map(|k| env(k).filter(|v| !v.trim().is_empty())) + }; + cfg.access_key_id = cfg.access_key_id.or_else(|| first(&["AWS_ACCESS_KEY_ID"])); + cfg.secret_access_key = cfg + .secret_access_key + .or_else(|| first(&["AWS_SECRET_ACCESS_KEY"])); + cfg.session_token = cfg.session_token.or_else(|| first(&["AWS_SESSION_TOKEN"])); + cfg.region = cfg + .region + .or_else(|| first(&["AWS_REGION", "AWS_DEFAULT_REGION"])); + cfg.endpoint = cfg + .endpoint + .or_else(|| first(&["AWS_ENDPOINT_URL_S3", "AWS_ENDPOINT_URL"])); + cfg +} + +/// Settings as `object_store` key/value options. Unrecognized keys are +/// ignored by `parse_url_opts`, so the same list is safe to pass for a +/// `file://` URL as for `s3://`. +fn store_options(cfg: &S3Settings) -> Vec<(&'static str, String)> { + fn push(opts: &mut Vec<(&'static str, String)>, k: &'static str, v: &Option) { + if let Some(v) = v.as_deref().map(str::trim).filter(|v| !v.is_empty()) { + opts.push((k, v.to_string())); + } + } + + let mut opts: Vec<(&'static str, String)> = Vec::new(); + + // Credentials come from `resolve_credentials`, not straight off + // `cfg` — a stored key is only one of the places they can live, and + // the common laptop case is an AWS profile we had to go find. + let resolved = cfg.resolved_credentials(); + if let Some(c) = resolved.as_ref().and_then(|r| r.credentials.as_ref()) { + opts.push(("aws_access_key_id", c.access_key_id.clone())); + opts.push(("aws_secret_access_key", c.secret_access_key.clone())); + if let Some(t) = &c.session_token { + opts.push(("aws_session_token", t.clone())); + } + } + + push(&mut opts, "aws_endpoint", &cfg.endpoint); + let region = cfg + .region + .clone() + .or_else(|| resolved.as_ref().and_then(|r| r.region.clone())) + .unwrap_or_else(|| DEFAULT_REGION.to_string()); + opts.push(("aws_region", region)); + + if let Some(v) = cfg.virtual_hosted_style { + opts.push(("aws_virtual_hosted_style_request", v.to_string())); + } + // A plaintext endpoint is a deliberate choice (MinIO on localhost, + // a test fixture); object_store refuses http:// unless told. + if cfg + .endpoint + .as_deref() + .is_some_and(|e| e.starts_with("http://")) + { + opts.push(("aws_allow_http", "true".to_string())); + } + opts +} + +// ── Locations ─────────────────────────────────────────────────────────── + +/// A single object in object storage. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ObjectUri { + url: Url, +} + +impl std::fmt::Display for ObjectUri { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&friendly(&self.url)) + } +} + +/// True for anything `path resume` / `p import` should route to object +/// storage rather than to Pathbase or the local cache. +pub(crate) fn looks_like_object_uri(s: &str) -> bool { + SCHEMES.iter().any(|p| s.starts_with(&format!("{p}://"))) +} + +impl ObjectUri { + /// Parse a full object reference. A container with no key names a + /// place, not a document, so it's rejected here — the share side + /// goes through [`Destination`] instead. + pub(crate) fn parse(raw: &str) -> Result { + let url = parse_location(raw)?; + if url.path().trim_matches('/').is_empty() { + bail!( + "`{raw}` names a location but no object key \ + (expected s3://bucket/path/to/doc.json)" + ); + } + Ok(ObjectUri { url }) + } + + /// The cache id a download of this object lands at, e.g. + /// `s3-my-bucket-traces_claude-abc`. + pub(crate) fn cache_id(&self) -> String { + let source = match self.url.scheme() { + "s3a" => "s3", + other => other, + }; + let host = self.url.host_str().unwrap_or_default(); + let key = self.url.path().trim_matches('/'); + let inner = if host.is_empty() { + key.to_string() + } else { + format!("{host}-{key}") + }; + crate::cache::make_id(source, &inner) + } + + /// Download the object as UTF-8 text. + pub(crate) fn get(&self, cfg: &S3Settings) -> Result { + let (store, path) = open(&self.url, cfg)?; + let bytes = block_on(async { + let result = store.get(&path).await?; + result.bytes().await + }) + .map_err(|e| explain_location(e, "read", &self.to_string()))?; + String::from_utf8(bytes.to_vec()).with_context(|| format!("{self} is not valid UTF-8")) + } + + /// Upload `body` to the object, overwriting any existing one. + /// + /// Overwrite is intentional: the object name is a pure function of + /// the document, so re-sharing a session that has grown replaces + /// its own object rather than accumulating near-duplicates. + pub(crate) fn put(&self, cfg: &S3Settings, body: &[u8]) -> Result<()> { + let (store, path) = open(&self.url, cfg)?; + let payload = object_store::PutPayload::from(body.to_vec()); + block_on(store.put(&path, payload)) + .map(|_| ()) + .map_err(|e| explain_location(e, "write", &self.to_string())) + } +} + +fn open(url: &Url, cfg: &S3Settings) -> Result<(Box, object_store::path::Path)> { + open_with(url, cfg, Vec::new()) +} + +fn open_with( + url: &Url, + cfg: &S3Settings, + extra: Vec<(&'static str, String)>, +) -> Result<(Box, object_store::path::Path)> { + let mut opts = store_options(cfg); + opts.extend(extra); + object_store::parse_url_opts(url, opts).with_context(|| format!("open {}", friendly(url))) +} + +/// Strip `object_store`'s internals out of an error message. +/// +/// Its transport errors carry a retry epilogue — "after 10 retries, +/// max_retries: 10, retry_timeout: 180s" — plus a `Generic S3 error:` +/// prefix. Neither tells a user anything actionable, and both bury the +/// part that does. +fn terse(err: &object_store::Error) -> String { + let msg = err.to_string(); + let msg = msg.split(", after ").next().unwrap_or(&msg); + msg.trim_start_matches("Generic S3 error: ") + .trim_end_matches([' ', '-']) + .to_string() +} + +/// Where `path share` writes when the target is object storage: a +/// bucket-or-folder base that object keys hang off. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct Destination { + base: Url, +} + +impl std::fmt::Display for Destination { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(friendly(&self.base).trim_end_matches('/')) + } +} + +/// One object found by [`Destination::list`]. +#[derive(Debug, Clone)] +pub(crate) struct ObjectEntry { + pub uri: ObjectUri, + /// Filename without the `.json` extension — for legible names this + /// is `--`, which is the whole point. + pub stem: String, + pub size: u64, + pub modified: Option>, +} + +impl Destination { + /// Parse a user-supplied destination. See [`parse_location`] for + /// how a scheme-less value is read. + pub(crate) fn parse(raw: &str) -> Result { + Ok(Destination { + base: parse_location(raw)?, + }) + } + + /// The `.json` objects sitting directly under this destination, + /// newest first. + /// + /// Deliberately non-recursive: a destination is a place you share + /// *to*, so its immediate contents are what a picker should offer. + /// Nothing is downloaded — legible object names carry enough for a + /// picker row, which is exactly why they're worth the length. + pub(crate) fn list(&self, cfg: &S3Settings) -> Result> { + let (store, prefix) = open(&self.base, cfg)?; + let listed = block_on(store.list_with_delimiter(Some(&prefix))) + .map_err(|e| explain_location(e, "list", &friendly(&self.base)))?; + + let mut out: Vec = listed + .objects + .into_iter() + .filter(|m| m.location.as_ref().ends_with(".json")) + .map(|m| { + let name = m + .location + .filename() + .unwrap_or_default() + .trim_end_matches(".json") + .to_string(); + let mut url = self.base.clone(); + let base_path = url.path().trim_end_matches('/').to_string(); + url.set_path(&format!("{base_path}/{name}.json")); + ObjectEntry { + uri: ObjectUri { url }, + stem: name, + size: m.size, + modified: Some(m.last_modified), + } + }) + .collect(); + // Newest first: the session you want is nearly always recent. + out.sort_by(|a, b| b.modified.cmp(&a.modified).then(a.stem.cmp(&b.stem))); + Ok(out) + } + + pub(crate) fn uri_for(&self, name: &ObjectName) -> ObjectUri { + let mut url = self.base.clone(); + let base_path = url.path().trim_end_matches('/').to_string(); + url.set_path(&format!("{base_path}/{}.json", name.0)); + ObjectUri { url } + } +} + +// ── Naming ────────────────────────────────────────────────────────────── + +/// What a shared document is called in the destination. +/// +/// `--`, e.g. +/// `2026-08-07-add-s3-support-to-share-claude-6f2a1c9e`. +/// +/// Two requirements pull in opposite directions and both are load-bearing: +/// +/// - **Stable.** Every component is a pure function of the document, so +/// re-sharing a session that has grown overwrites its own object +/// instead of leaving a trail of near-duplicates. +/// - **Legible.** A destination is a folder someone will open, or a +/// bucket someone will page through. `claude-6f2a1c9e.json` tells +/// them nothing; the date sorts chronologically under a plain +/// lexicographic listing, and the slug says which session it is. +/// +/// Legibility also buys the picker: `path resume ` builds +/// its rows from names alone, so browsing a hundred shared sessions +/// costs one list request and zero downloads. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct ObjectName(String); + +impl std::fmt::Display for ObjectName { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +impl ObjectName { + /// Longest slug we'll put in a name. Long enough to recognize a + /// session, short enough that the cache id stays visible in a + /// terminal-width listing. + const SLUG_MAX: usize = 48; + + pub(crate) fn new(cache_id: &str, date: Option<&str>, title: Option<&str>) -> Self { + let mut parts: Vec = Vec::new(); + if let Some(d) = date.map(slugify).filter(|d| !d.is_empty()) { + parts.push(d); + } + if let Some(t) = title.map(slugify).filter(|t| !t.is_empty()) { + parts.push(truncate_slug(&t, Self::SLUG_MAX)); + } + parts.push(slugify(cache_id)); + ObjectName(parts.join("-")) + } + + /// The name for a document with no usable metadata — the cache id + /// alone, which is what the whole scheme degrades to. + #[cfg(test)] + pub(crate) fn bare(cache_id: &str) -> Self { + Self::new(cache_id, None, None) + } +} + +/// Lowercase, ASCII-alphanumeric, single dashes, no leading/trailing +/// dash. Deliberately lossy — this is a filename, not a title. +fn slugify(raw: &str) -> String { + let mut out = String::with_capacity(raw.len()); + let mut pending_dash = false; + for ch in raw.chars() { + if ch.is_ascii_alphanumeric() { + if pending_dash && !out.is_empty() { + out.push('-'); + } + pending_dash = false; + out.push(ch.to_ascii_lowercase()); + } else { + pending_dash = true; + } + } + out +} + +/// Truncate on a dash boundary so a name never ends mid-word. +fn truncate_slug(slug: &str, max: usize) -> String { + if slug.len() <= max { + return slug.to_string(); + } + let cut = &slug[..max]; + match cut.rfind('-') { + Some(i) if i > 0 => cut[..i].to_string(), + _ => cut.to_string(), + } +} + +/// Name a document for a destination, reading the date and topic out of +/// the document itself so `share` and `p export object` agree without +/// either of them having to know where the document came from. +pub(crate) fn name_for(doc: &toolpath::v1::Graph, cache_id: &str) -> ObjectName { + let path = doc.paths.iter().find_map(|p| match p { + toolpath::v1::PathOrRef::Path(p) => Some(p.as_ref()), + toolpath::v1::PathOrRef::Ref(_) => None, + }); + let Some(path) = path else { + return ObjectName::new(cache_id, None, None); + }; + + // Earliest step wins: a session is dated when it started, so the + // name doesn't move as the conversation grows. + let date = path + .steps + .iter() + .map(|s| s.step.timestamp.as_str()) + .min() + .and_then(|ts| ts.split('T').next()) + .map(str::to_string); + + ObjectName::new(cache_id, date.as_deref(), topic_of(path).as_deref()) +} + +/// [`name_for`] against the serialized document — the exact bytes about +/// to be uploaded, so the name always describes what actually lands. +/// Degrades to the bare cache id if the body doesn't parse, because a +/// worse name is better than a failed share. +pub(crate) fn name_for_body(body: &str, cache_id: &str) -> ObjectName { + match toolpath::v1::Graph::from_json(body) { + Ok(doc) => name_for(&doc, cache_id), + Err(_) => ObjectName::new(cache_id, None, None), + } +} + +/// The first user prompt, which is what a session is *about*. +/// +/// Falls back to `meta.title`, but only when it looks like a real +/// title: `derive_path` synthesizes `" session: "` when +/// it has nothing better, and repeating the id in the slug would waste +/// the legible half of the name. +fn topic_of(path: &toolpath::v1::Path) -> Option { + for step in &path.steps { + for change in path_changes(step) { + let Some(structural) = &change.structural else { + continue; + }; + if structural.change_type != "conversation.append" { + continue; + } + let role = structural.extra.get("role").and_then(|v| v.as_str()); + if role != Some("user") { + continue; + } + if let Some(text) = structural + .extra + .get("text") + .and_then(|v| v.as_str()) + .map(str::trim) + .filter(|t| !t.is_empty()) + { + return Some(text.to_string()); + } + } + } + path.meta + .as_ref() + .and_then(|m| m.title.as_deref()) + .filter(|t| !t.contains(" session: ")) + .map(str::to_string) +} + +/// Steps hold their changes in a map; iteration order is arbitrary, so +/// sort by artifact key to keep naming deterministic across runs. +fn path_changes(step: &toolpath::v1::Step) -> Vec<&toolpath::v1::ArtifactChange> { + let mut keys: Vec<&String> = step.change.keys().collect(); + keys.sort(); + keys.into_iter() + .filter_map(|k| step.change.get(k)) + .collect() +} + +/// Parse a user-supplied location into a URL. +/// +/// A value carrying a scheme is taken at its word. A scheme-less value +/// is a **local filesystem path** — `~/traces`, `./out`, `/srv/traces` +/// — expanded, made absolute, and turned into a `file://` URL. Bare +/// strings mean folders rather than buckets because that is what people +/// type when designating a directory; an S3 bucket is named with an +/// explicit `s3://`, which is unambiguous and self-documenting. +/// +/// A *bare relative* path (`my-bucket/traces`) is rejected. It is the +/// one shape that is genuinely ambiguous — overwhelmingly a bucket name +/// typed from memory — and silently resolving it against the current +/// directory would create `./my-bucket/traces` and report success. +/// `./my-bucket/traces` says "yes, relative, I meant it". +fn parse_location(raw: &str) -> Result { + let raw = raw.trim(); + if raw.is_empty() { + bail!("empty location"); + } + + if !raw.contains("://") && is_ambiguously_relative(raw) { + bail!( + "`{raw}` is ambiguous: a scheme-less location is a local path, but this \ + one is relative to the current directory.\n \ + s3://{raw} — if you meant an S3 bucket\n \ + ./{raw} — if you really meant a folder here" + ); + } + + if raw.contains("://") { + let url = Url::parse(raw).with_context(|| format!("`{raw}` is not a valid URL"))?; + if !SCHEMES.contains(&url.scheme()) { + bail!( + "unsupported location scheme `{}://` (expected one of: {})", + url.scheme(), + SCHEMES + .iter() + .map(|s| format!("{s}://")) + .collect::>() + .join(", ") + ); + } + if matches!(url.scheme(), "s3" | "s3a") && url.host_str().unwrap_or_default().is_empty() { + bail!("`{raw}` has no bucket (expected s3://bucket/prefix)"); + } + return Ok(url); + } + + let expanded = expand_tilde(raw); + let absolute = std::path::absolute(&expanded) + .with_context(|| format!("resolve `{}` to an absolute path", expanded.display()))?; + Url::from_directory_path(&absolute).map_err(|()| { + anyhow!( + "`{raw}` is neither a URL nor a usable filesystem path \ + (for an S3 bucket, write it as s3://{raw})" + ) + }) +} + +/// True for a path that is relative *and* doesn't say so explicitly. +/// `./x` and `../x` are deliberate; `x` and `x/y` are the trap. +fn is_ambiguously_relative(raw: &str) -> bool { + !(raw.starts_with('/') + || raw.starts_with('~') + || raw.starts_with("./") + || raw.starts_with("../") + || raw == "." + || raw == ".." + // Windows: `C:\…` and `\\server\share`. + || raw.starts_with('\\') + || raw.as_bytes().get(1) == Some(&b':')) +} + +/// Expand a leading `~/`. Shells normally do this, but a quoted or +/// config-file value arrives literal, and `object_store` would happily +/// create a directory actually named `~`. +fn expand_tilde(raw: &str) -> PathBuf { + let Some(rest) = raw.strip_prefix("~/") else { + return PathBuf::from(raw); + }; + match std::env::var_os("HOME") { + Some(home) => PathBuf::from(home).join(rest), + None => PathBuf::from(raw), + } +} + +/// Render a location for humans: a `file://` URL shows as the plain +/// path it names, which is both shorter and directly pasteable into +/// `path resume`. Everything else shows as its URL. +fn friendly(url: &Url) -> String { + if url.scheme() == "file" + && let Ok(p) = url.to_file_path() + { + return p.to_string_lossy().into_owned(); + } + url.as_str().to_string() +} + +/// Turn an `object_store` error into something a user can act on. Its +/// `NotFound` and `Unauthenticated` variants are the two that matter: +/// the first usually means a typo'd key, the second an unconfigured or +/// stale credential. +fn explain_location(err: object_store::Error, verb: &str, location: &str) -> anyhow::Error { + match err { + object_store::Error::NotFound { .. } => anyhow!("{location} not found"), + object_store::Error::Unauthenticated { .. } + | object_store::Error::PermissionDenied { .. } => { + anyhow!( + "not authorized to {verb} {location}. Run `path auth s3 login` to store \ + credentials, or check the bucket policy for the ones you have." + ) + } + e => anyhow!("failed to {verb} {location}: {}", terse(&e)), + } +} + +/// `object_store` is async; the rest of path-cli is sync. Same tunnel +/// the Pathbase client uses, so both share one runtime. +fn block_on(f: F) -> F::Output { + crate::cmd_pathbase::block_on(f) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_s3_uris() { + let u = ObjectUri::parse("s3://my-bucket/traces/claude-abc.json").unwrap(); + assert_eq!(u.to_string(), "s3://my-bucket/traces/claude-abc.json"); + } + + #[test] + fn container_without_a_key_is_not_an_object() { + let err = ObjectUri::parse("s3://my-bucket").unwrap_err().to_string(); + assert!(err.contains("no object key"), "{err}"); + } + + #[test] + fn looks_like_object_uri_only_matches_known_schemes() { + assert!(looks_like_object_uri("s3://b/k")); + assert!(looks_like_object_uri("s3a://b/k")); + assert!(looks_like_object_uri("file:///tmp/k.json")); + // https belongs to Pathbase; a bare id belongs to the cache. + assert!(!looks_like_object_uri("https://pathbase.dev/a/b/c")); + assert!(!looks_like_object_uri("claude-abc")); + } + + #[test] + fn unsupported_scheme_lists_the_supported_ones() { + let err = ObjectUri::parse("gs://bucket/key.json") + .unwrap_err() + .to_string(); + assert!(err.contains("s3://"), "{err}"); + } + + // ── Destinations ───────────────────────────────────────────────── + + #[test] + fn s3_destination_keys_on_the_cache_id() { + let d = Destination::parse("s3://bkt/pre/fix").unwrap(); + assert_eq!( + d.uri_for(&ObjectName::bare("claude-abc")).to_string(), + "s3://bkt/pre/fix/claude-abc.json" + ); + } + + #[test] + fn s3_destination_without_a_prefix_writes_at_the_bucket_root() { + let d = Destination::parse("s3://bkt").unwrap(); + assert_eq!( + d.uri_for(&ObjectName::bare("claude-abc")).to_string(), + "s3://bkt/claude-abc.json" + ); + } + + #[test] + fn a_bare_path_is_a_local_folder_not_a_bucket() { + let d = Destination::parse("/srv/traces").unwrap(); + assert_eq!(d.to_string(), "/srv/traces"); + assert_eq!( + d.uri_for(&ObjectName::bare("claude-abc")).to_string(), + "/srv/traces/claude-abc.json" + ); + } + + #[test] + fn an_explicitly_relative_path_is_made_absolute_so_a_stored_default_is_stable() { + let d = Destination::parse("./out").unwrap(); + let expected = std::path::absolute("./out").unwrap(); + assert_eq!(d.to_string(), expected.to_string_lossy()); + } + + #[test] + fn a_bare_relative_path_is_rejected_as_ambiguous() { + // The trap this guards: someone types a bucket name from memory + // and gets ./my-bucket/traces created under their cwd, with the + // share reporting success. + let err = Destination::parse("my-bucket/traces") + .unwrap_err() + .to_string(); + assert!(err.contains("s3://my-bucket/traces"), "{err}"); + assert!(err.contains("./my-bucket/traces"), "{err}"); + + // A single bare word is the same mistake. + assert!(Destination::parse("my-bucket").is_err()); + // Saying "relative, I meant it" is accepted. + assert!(Destination::parse("./my-bucket/traces").is_ok()); + assert!(Destination::parse("../sibling").is_ok()); + } + + #[test] + fn memory_urls_are_rejected() { + // A fresh store per process: anything "shared" there is gone + // before the command exits. + let err = Destination::parse("memory:///x").unwrap_err().to_string(); + assert!(err.contains("unsupported location scheme"), "{err}"); + } + + #[test] + fn a_tilde_path_expands_against_home() { + let home = std::env::var("HOME").unwrap(); + let d = Destination::parse("~/traces").unwrap(); + assert_eq!(d.to_string(), format!("{home}/traces")); + } + + #[test] + fn a_file_url_and_the_equivalent_bare_path_agree() { + let from_path = Destination::parse("/srv/traces").unwrap(); + let from_url = Destination::parse("file:///srv/traces").unwrap(); + assert_eq!( + from_path.uri_for(&ObjectName::bare("x")).to_string(), + from_url.uri_for(&ObjectName::bare("x")).to_string() + ); + } + + #[test] + fn local_destinations_display_as_plain_paths() { + // The printed form is what a user pastes into `path resume`, + // and `path resume /abs/path.json` already works. + let uri = Destination::parse("/srv/traces") + .unwrap() + .uri_for(&ObjectName::bare("claude-abc")); + assert_eq!(uri.to_string(), "/srv/traces/claude-abc.json"); + } + + #[test] + fn cache_id_flattens_the_key() { + let uri = ObjectUri::parse("s3://bkt/traces/claude-abc.json").unwrap(); + assert_eq!(uri.cache_id(), "s3-bkt-traces_claude-abc"); + // s3a is the same store under a different scheme spelling, so + // it must not fork the cache. + let alias = ObjectUri::parse("s3a://bkt/traces/claude-abc.json").unwrap(); + assert_eq!(alias.cache_id(), uri.cache_id()); + } + + // ── S3 settings ────────────────────────────────────────────────── + + #[test] + fn env_fills_only_the_gaps() { + let stored = S3Settings { + region: Some("eu-west-1".to_string()), + ..Default::default() + }; + let merged = merge_env(stored, |k| match k { + "AWS_ACCESS_KEY_ID" => Some("AK".to_string()), + "AWS_SECRET_ACCESS_KEY" => Some("SK".to_string()), + "AWS_REGION" => Some("us-west-2".to_string()), + _ => None, + }); + assert_eq!(merged.access_key_id.as_deref(), Some("AK")); + // Stored wins over env. + assert_eq!(merged.region.as_deref(), Some("eu-west-1")); + } + + #[test] + fn blank_env_values_are_ignored() { + let merged = merge_env(S3Settings::default(), |k| match k { + "AWS_ACCESS_KEY_ID" => Some(" ".to_string()), + _ => None, + }); + assert!(merged.access_key_id.is_none()); + } + + #[test] + fn store_options_carry_credentials_and_endpoint() { + let opts = store_options(&S3Settings { + access_key_id: Some("AK".to_string()), + secret_access_key: Some("SK".to_string()), + endpoint: Some("http://127.0.0.1:9000".to_string()), + ..Default::default() + }); + let get = |k: &str| { + opts.iter() + .find(|(key, _)| *key == k) + .map(|(_, v)| v.as_str()) + }; + assert_eq!(get("aws_access_key_id"), Some("AK")); + assert_eq!(get("aws_secret_access_key"), Some("SK")); + assert_eq!(get("aws_endpoint"), Some("http://127.0.0.1:9000")); + // Plaintext endpoints have to be opted into explicitly. + assert_eq!(get("aws_allow_http"), Some("true")); + assert_eq!(get("aws_region"), Some(DEFAULT_REGION)); + } + + #[test] + fn https_endpoint_does_not_allow_http() { + let opts = store_options(&S3Settings { + endpoint: Some("https://minio.example".to_string()), + ..Default::default() + }); + assert!(!opts.iter().any(|(k, _)| *k == "aws_allow_http")); + } + + #[test] + fn stored_settings_round_trip_through_disk() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("s3.json"); + let cfg = S3Settings { + region: Some("us-east-2".to_string()), + access_key_id: Some("AK".to_string()), + secret_access_key: Some("SK".to_string()), + ..Default::default() + }; + store(&path, &cfg).unwrap(); + assert_eq!(load_stored(&path).unwrap().unwrap(), cfg); + + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(&path).unwrap().permissions().mode(); + assert_eq!(mode & 0o777, 0o600); + } + + clear(&path).unwrap(); + assert!(load_stored(&path).unwrap().is_none()); + // Clearing settings that aren't there is not an error. + clear(&path).unwrap(); + } + + // ── Round trips against a local folder ─────────────────────────── + + #[test] + fn put_then_get_round_trips() { + let dir = tempfile::tempdir().unwrap(); + let dest = Destination::parse(&dir.path().to_string_lossy()).unwrap(); + let cfg = S3Settings::default(); + let uri = dest.uri_for(&ObjectName::bare("claude-abc")); + let body = br#"{"graph":{"id":"g"},"paths":[]}"#; + + uri.put(&cfg, body).unwrap(); + assert_eq!(uri.get(&cfg).unwrap(), String::from_utf8_lossy(body)); + assert!(dir.path().join("claude-abc.json").is_file()); + } + + #[test] + fn put_overwrites_an_existing_object() { + let dir = tempfile::tempdir().unwrap(); + let dest = Destination::parse(&dir.path().to_string_lossy()).unwrap(); + let cfg = S3Settings::default(); + let uri = dest.uri_for(&ObjectName::bare("claude-abc")); + + uri.put(&cfg, b"{\"v\":1}").unwrap(); + uri.put(&cfg, b"{\"v\":2}").unwrap(); + assert_eq!(uri.get(&cfg).unwrap(), "{\"v\":2}"); + } + + #[test] + fn nested_prefixes_are_created_on_write() { + let dir = tempfile::tempdir().unwrap(); + let dest = Destination::parse(&format!("{}/a/b/c", dir.path().display())).unwrap(); + dest.uri_for(&ObjectName::bare("claude-abc")) + .put(&S3Settings::default(), b"{}") + .unwrap(); + assert!(dir.path().join("a/b/c/claude-abc.json").is_file()); + } + + #[test] + fn missing_object_says_not_found() { + let dir = tempfile::tempdir().unwrap(); + let uri = Destination::parse(&dir.path().to_string_lossy()) + .unwrap() + .uri_for(&ObjectName::bare("nope")); + let err = uri.get(&S3Settings::default()).unwrap_err().to_string(); + assert!(err.contains("not found"), "{err}"); + assert!(err.contains("nope.json"), "{err}"); + } + + // ── Naming ─────────────────────────────────────────────────────── + + /// A one-step agent document whose first user turn is `prompt`. + fn doc_with(prompt: &str, timestamp: &str) -> toolpath::v1::Graph { + let body = serde_json::json!({ + "graph": { "id": "g1" }, + "paths": [{ + "path": { "id": "p1", "head": "s1" }, + "steps": [{ + "step": { + "id": "s1", "parents": [], + "actor": "agent:claude-code", + "timestamp": timestamp + }, + "change": { "claude-code://s": { "structural": { + "type": "conversation.append", + "role": "user", + "text": prompt + }}} + }] + }] + }); + toolpath::v1::Graph::from_json(&body.to_string()).unwrap() + } + + #[test] + fn a_name_leads_with_the_date_and_topic() { + let doc = doc_with("Add S3 support to share", "2026-08-07T09:15:00Z"); + assert_eq!( + name_for(&doc, "claude-abc123").to_string(), + "2026-08-07-add-s3-support-to-share-claude-abc123" + ); + } + + #[test] + fn a_name_is_stable_as_the_session_grows() { + // The date comes from the *earliest* step, so appending turns + // can't move the object and leave a duplicate behind. + let short = doc_with("Fix the parser", "2026-08-07T09:15:00Z"); + let name = name_for(&short, "claude-abc"); + + let mut grown = short.clone(); + if let toolpath::v1::PathOrRef::Path(p) = &mut grown.paths[0] { + let mut later = p.steps[0].clone(); + later.step.id = "s2".to_string(); + later.step.timestamp = "2026-08-09T18:00:00Z".to_string(); + p.steps.push(later); + } + assert_eq!(name_for(&grown, "claude-abc"), name); + } + + #[test] + fn a_long_prompt_is_truncated_on_a_word_boundary() { + let doc = doc_with( + "Add support to share and resume to and from S3 and a way to configure credentials", + "2026-08-07T00:00:00Z", + ); + let name = name_for(&doc, "claude-abc").to_string(); + assert!( + name.starts_with("2026-08-07-add-support-to-share"), + "{name}" + ); + assert!(name.ends_with("-claude-abc"), "{name}"); + assert!(!name.contains("--"), "no empty slug segments: {name}"); + } + + #[test] + fn a_prompt_of_pure_punctuation_degrades_to_date_and_id() { + let doc = doc_with("!!! ???", "2026-08-07T00:00:00Z"); + assert_eq!( + name_for(&doc, "claude-abc").to_string(), + "2026-08-07-claude-abc" + ); + } + + #[test] + fn a_synthesized_title_is_not_worth_slugging() { + // `derive_path` writes "claude-code session: abc" when it has + // nothing better; repeating the id would waste the legible half + // of the name. + let body = serde_json::json!({ + "graph": { "id": "g1" }, + "paths": [{ + "path": { "id": "p1", "head": "s1" }, + "meta": { "title": "claude-code session: abc123" }, + "steps": [{ + "step": { "id": "s1", "parents": [], "actor": "agent:claude-code", + "timestamp": "2026-08-07T00:00:00Z" }, + "change": { "f": { "structural": { "type": "file.edit" } } } + }] + }] + }); + let doc = toolpath::v1::Graph::from_json(&body.to_string()).unwrap(); + assert_eq!( + name_for(&doc, "claude-abc").to_string(), + "2026-08-07-claude-abc" + ); + } + + #[test] + fn an_unparseable_body_still_gets_a_name() { + // A worse name beats a failed share. + assert_eq!( + name_for_body("not json", "claude-abc").to_string(), + "claude-abc" + ); + } + + // ── Listing ────────────────────────────────────────────────────── + + #[test] + fn listing_returns_shared_documents_newest_first() { + let dir = tempfile::tempdir().unwrap(); + let dest = Destination::parse(&dir.path().to_string_lossy()).unwrap(); + let cfg = S3Settings::default(); + + for name in ["2026-08-01-older-claude-a", "2026-08-09-newer-claude-b"] { + dest.uri_for(&ObjectName::bare(name)) + .put(&cfg, b"{}") + .unwrap(); + // Distinct mtimes; the local backend stamps on write. + std::thread::sleep(std::time::Duration::from_millis(10)); + } + // Noise that isn't a shared document. + std::fs::write(dir.path().join("README.txt"), "hi").unwrap(); + + let entries = dest.list(&cfg).unwrap(); + assert_eq!(entries.len(), 2, "{entries:?}"); + assert_eq!(entries[0].stem, "2026-08-09-newer-claude-b"); + assert_eq!(entries[1].stem, "2026-08-01-older-claude-a"); + assert!(entries[0].uri.to_string().ends_with(".json")); + assert!(entries[0].size > 0); + } + + #[test] + fn listing_does_not_recurse_into_sub_prefixes() { + // A destination is a place you share *to*; its immediate + // contents are what a picker should offer. + let dir = tempfile::tempdir().unwrap(); + let dest = Destination::parse(&dir.path().to_string_lossy()).unwrap(); + let cfg = S3Settings::default(); + dest.uri_for(&ObjectName::bare("here")) + .put(&cfg, b"{}") + .unwrap(); + + let nested = Destination::parse(&format!("{}/deeper", dir.path().display())).unwrap(); + nested + .uri_for(&ObjectName::bare("there")) + .put(&cfg, b"{}") + .unwrap(); + + let entries = dest.list(&cfg).unwrap(); + assert_eq!(entries.len(), 1); + assert_eq!(entries[0].stem, "here"); + } + + #[test] + fn listing_an_empty_destination_is_not_an_error() { + let dir = tempfile::tempdir().unwrap(); + let dest = Destination::parse(&dir.path().to_string_lossy()).unwrap(); + assert!(dest.list(&S3Settings::default()).unwrap().is_empty()); + } +} diff --git a/crates/path-cli/tests/object_storage.rs b/crates/path-cli/tests/object_storage.rs new file mode 100644 index 00000000..69b14456 --- /dev/null +++ b/crates/path-cli/tests/object_storage.rs @@ -0,0 +1,351 @@ +//! Integration tests for object storage: `p export object`, +//! `p import object`, and `path auth s3`. +//! +//! Every test runs against a local folder — the same `object_store` +//! code path an `s3://` bucket takes, minus the network. That's the +//! point of making folders first-class: the plumbing under test +//! (destination parsing, naming, upload, download, cache landing) is +//! backend-independent, so exercising it locally covers the S3 case +//! without credentials or a mock endpoint. + +#![cfg(not(target_os = "emscripten"))] + +use assert_cmd::Command; +use predicates::prelude::*; +use std::path::Path; + +fn cmd(config_dir: &Path) -> Command { + let mut c = Command::cargo_bin("path").unwrap(); + c.env("TOOLPATH_CONFIG_DIR", config_dir); + for k in [ + "AWS_ACCESS_KEY_ID", + "AWS_SECRET_ACCESS_KEY", + "AWS_SESSION_TOKEN", + "AWS_REGION", + "AWS_DEFAULT_REGION", + "AWS_ENDPOINT_URL", + "AWS_ENDPOINT_URL_S3", + "AWS_PROFILE", + ] { + c.env_remove(k); + } + // Credential resolution reads `~/.aws`, so point it at files that + // don't exist. Without this a developer's real default profile + // leaks in and the "no credentials" cases make real network calls. + c.env( + "AWS_SHARED_CREDENTIALS_FILE", + "/nonexistent/toolpath-test/credentials", + ); + c.env("AWS_CONFIG_FILE", "/nonexistent/toolpath-test/config"); + c +} + +/// A minimal single-step agent document, written to `dir/doc.json`. +fn write_doc(dir: &Path) -> std::path::PathBuf { + let body = serde_json::json!({ + "graph": { "id": "g1" }, + "paths": [{ + "path": { "id": "p1", "head": "s1" }, + "steps": [{ + "step": { + "id": "s1", "parents": [], + "actor": "agent:claude-code", + "timestamp": "2026-01-01T00:00:00Z" + }, + "change": { "claude-code://object-int": { "structural": { + "type": "conversation.append", "role": "user", "text": "hello" + }}} + }] + }] + }); + let p = dir.join("doc.json"); + std::fs::write(&p, serde_json::to_string(&body).unwrap()).unwrap(); + p +} + +// ── p export object / p import object ─────────────────────────────── + +#[test] +fn export_then_import_round_trips_through_a_folder() { + let config = tempfile::tempdir().unwrap(); + let work = tempfile::tempdir().unwrap(); + let folder = tempfile::tempdir().unwrap(); + let doc = write_doc(work.path()); + + let out = cmd(config.path()) + .args(["p", "export", "object"]) + .args(["--input", doc.to_str().unwrap()]) + .args(["--to", &folder.path().to_string_lossy()]) + .assert() + .success(); + let uri = String::from_utf8(out.get_output().stdout.clone()) + .unwrap() + .trim() + .to_string(); + + // Legible name: date and topic lead, cache id trails. The fixture + // is a 2026-01-01 session whose first prompt is "hello". + assert!( + uri.ends_with("/2026-01-01-hello-doc.json"), + "unexpected location: {uri}" + ); + assert!(folder.path().join("2026-01-01-hello-doc.json").is_file()); + + cmd(config.path()) + .args(["p", "import", "object", &format!("file://{uri}")]) + .assert() + .success(); + + let ids: Vec = std::fs::read_dir(config.path().join("documents")) + .unwrap() + .map(|e| e.unwrap().file_name().to_string_lossy().into_owned()) + .collect(); + assert_eq!(ids.len(), 1, "expected one cached doc, got {ids:?}"); + assert!(ids[0].starts_with("file-"), "unexpected cache id: {ids:?}"); +} + +#[test] +fn re_exporting_a_session_overwrites_its_own_object() { + // Every part of the name is a pure function of the document, so a + // re-share must not leave a trail of near-duplicates. + let config = tempfile::tempdir().unwrap(); + let work = tempfile::tempdir().unwrap(); + let folder = tempfile::tempdir().unwrap(); + let doc = write_doc(work.path()); + + for _ in 0..2 { + cmd(config.path()) + .args(["p", "export", "object"]) + .args(["--input", doc.to_str().unwrap()]) + .args(["--to", &folder.path().to_string_lossy()]) + .assert() + .success(); + } + + let objects: Vec = std::fs::read_dir(folder.path()) + .unwrap() + .map(|e| e.unwrap().file_name().to_string_lossy().into_owned()) + .collect(); + assert_eq!(objects, vec!["2026-01-01-hello-doc.json".to_string()]); +} + +#[test] +fn the_s3_subcommand_alias_still_works() { + let config = tempfile::tempdir().unwrap(); + let work = tempfile::tempdir().unwrap(); + let folder = tempfile::tempdir().unwrap(); + let doc = write_doc(work.path()); + + cmd(config.path()) + .args(["p", "export", "s3"]) + .args(["--input", doc.to_str().unwrap()]) + .args(["--to", &folder.path().to_string_lossy()]) + .assert() + .success(); + assert!(folder.path().join("2026-01-01-hello-doc.json").is_file()); +} + +#[test] +fn a_bare_relative_destination_is_rejected_rather_than_creating_a_folder() { + // The trap: a bucket name typed from memory becomes ./my-bucket + // under the cwd, and the export reports success. + let config = tempfile::tempdir().unwrap(); + let work = tempfile::tempdir().unwrap(); + let doc = write_doc(work.path()); + + cmd(config.path()) + .current_dir(work.path()) + .args(["p", "export", "object"]) + .args(["--input", doc.to_str().unwrap()]) + .args(["--to", "my-bucket/traces"]) + .assert() + .failure() + .stderr(predicate::str::contains("s3://my-bucket/traces")) + .stderr(predicate::str::contains("./my-bucket/traces")); + + assert!( + !work.path().join("my-bucket").exists(), + "a rejected destination must not leave a directory behind" + ); +} + +#[test] +fn an_unsupported_scheme_is_rejected() { + let config = tempfile::tempdir().unwrap(); + let work = tempfile::tempdir().unwrap(); + let doc = write_doc(work.path()); + cmd(config.path()) + .args(["p", "export", "object"]) + .args(["--input", doc.to_str().unwrap()]) + .args(["--to", "gs://bucket/x"]) + .assert() + .failure() + .stderr(predicate::str::contains("s3://")); +} + +#[test] +fn import_object_reports_a_missing_object_clearly() { + let config = tempfile::tempdir().unwrap(); + let folder = tempfile::tempdir().unwrap(); + cmd(config.path()) + .args([ + "p", + "import", + "object", + &format!("file://{}/nope.json", folder.path().display()), + ]) + .assert() + .failure() + .stderr(predicate::str::contains("not found")); +} + +#[test] +fn import_object_rejects_a_non_toolpath_object() { + let config = tempfile::tempdir().unwrap(); + let folder = tempfile::tempdir().unwrap(); + std::fs::write(folder.path().join("junk.json"), "{\"hello\":1}").unwrap(); + cmd(config.path()) + .args([ + "p", + "import", + "object", + &format!("file://{}/junk.json", folder.path().display()), + ]) + .assert() + .failure() + .stderr(predicate::str::contains("not a toolpath document")); +} + +// ── path auth s3 ──────────────────────────────────────────────────── + +#[test] +fn auth_s3_login_stores_status_shows_and_logout_clears() { + let config = tempfile::tempdir().unwrap(); + + cmd(config.path()) + .args(["auth", "s3", "login"]) + .args(["--region", "eu-west-1"]) + .args(["--access-key-id", "AKIAEXAMPLE"]) + .args(["--secret-access-key", "supersecretvalue"]) + .assert() + .success() + .stdout(predicate::str::contains("S3 settings saved")); + + let stored = config.path().join("s3.json"); + assert!(stored.is_file(), "s3.json not written"); + + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(&stored).unwrap().permissions().mode(); + assert_eq!( + mode & 0o777, + 0o600, + "credentials must not be world-readable" + ); + } + + cmd(config.path()) + .args(["auth", "s3", "status"]) + .assert() + .success() + .stdout(predicate::str::contains("eu-west-1")) + .stdout(predicate::str::contains("AKIAEXAMPLE")) + // The secret is stored but never printed back in full. + .stdout(predicate::str::contains("supersecretvalue").not()) + .stdout(predicate::str::contains("****alue")) + // And status says which source a share would actually use. + .stdout(predicate::str::contains("credentials: stored by")); + + cmd(config.path()) + .args(["auth", "s3", "logout"]) + .assert() + .success() + .stdout(predicate::str::contains("cleared")); + assert!(!stored.exists()); +} + +#[test] +fn auth_s3_login_merges_into_the_existing_settings() { + let config = tempfile::tempdir().unwrap(); + cmd(config.path()) + .args(["auth", "s3", "login", "--access-key-id", "AKIAEXAMPLE"]) + .assert() + .success(); + // A later, narrower call must not wipe the key. + cmd(config.path()) + .args(["auth", "s3", "login", "--region", "us-west-2"]) + .assert() + .success(); + + let raw = std::fs::read_to_string(config.path().join("s3.json")).unwrap(); + assert!(raw.contains("AKIAEXAMPLE"), "{raw}"); + assert!(raw.contains("us-west-2"), "{raw}"); +} + +#[test] +fn auth_s3_status_marks_env_supplied_values() { + let config = tempfile::tempdir().unwrap(); + cmd(config.path()) + .args(["auth", "s3", "status"]) + .env("AWS_REGION", "ap-south-1") + .assert() + .success() + .stdout(predicate::str::contains("ap-south-1 (env)")); +} + +#[test] +fn auth_s3_login_without_a_terminal_or_flags_is_an_error() { + let config = tempfile::tempdir().unwrap(); + cmd(config.path()) + .args(["auth", "s3", "login"]) + .assert() + .failure() + .stderr(predicate::str::contains("Nothing to store")); +} + +// ── credential resolution, end to end ─────────────────────────────── + +#[test] +fn a_profile_is_picked_up_with_no_toolpath_configuration_at_all() { + // The whole point: someone who has run `aws configure` gets S3 + // access without telling us anything. + let config = tempfile::tempdir().unwrap(); + let aws = tempfile::tempdir().unwrap(); + let creds = aws.path().join("credentials"); + std::fs::write( + &creds, + "[default]\naws_access_key_id = AKIAPROFILE\naws_secret_access_key = s3cret\n", + ) + .unwrap(); + + cmd(config.path()) + .args(["auth", "s3", "status"]) + .env("AWS_SHARED_CREDENTIALS_FILE", &creds) + .assert() + .success() + .stdout(predicate::str::contains("credentials: profile `default`")); +} + +#[test] +fn no_credentials_anywhere_reports_the_instance_chain_not_a_failure() { + // On a server this is the correct answer, not an error. + let config = tempfile::tempdir().unwrap(); + cmd(config.path()) + .args(["auth", "s3", "status"]) + .assert() + .success() + .stdout(predicate::str::contains("EC2/ECS/EKS credential chain")); +} + +#[test] +fn an_unknown_profile_says_which_profile_and_how_to_list_them() { + let config = tempfile::tempdir().unwrap(); + cmd(config.path()) + .args(["auth", "s3", "status"]) + .env("AWS_PROFILE", "typo") + .assert() + .success() + .stdout(predicate::str::contains("no such profile")) + .stdout(predicate::str::contains("aws configure list-profiles")); +} diff --git a/crates/path-cli/tests/resume.rs b/crates/path-cli/tests/resume.rs index f751c40e..adb45a3a 100644 --- a/crates/path-cli/tests/resume.rs +++ b/crates/path-cli/tests/resume.rs @@ -275,6 +275,252 @@ fn cache_id_input_loads_and_projects() { assert_eq!(cap.args[0], "-r"); } +// ── Object-storage input ──────────────────────────────────────────── +// +// `file://` exercises the same `object_store` path an `s3://` bucket +// takes, so these cover the S3 resume flow without a network. + +/// Write `graph` into `bucket` as `.json` and return its URL. +fn seed_object(bucket: &std::path::Path, name: &str, graph: &toolpath::v1::Graph) -> String { + std::fs::write( + bucket.join(format!("{name}.json")), + graph.to_json().unwrap(), + ) + .unwrap(); + format!("file://{}/{name}.json", bucket.display()) +} + +#[test] +fn object_store_input_downloads_projects_and_caches() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let graph = toolpath::v1::Graph::from_path(make_convo_path( + "agent:claude-code", + "claude-code://resume-s3-int", + )); + let uri = seed_object(bucket.path(), "claude-abc", &graph); + + let recorder = RecordingExec::default(); + run_with_strategy( + ResumeArgs { + input: uri, + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: false, + force: false, + url: None, + }, + &recorder, + ) + .unwrap(); + + let cap = recorder.captured(); + assert_eq!(cap.binary, "claude"); + assert_eq!(cap.args[0], "-r"); + + // The download is cached, so a second resume costs no fetch. + let documents = std::path::PathBuf::from(std::env::var_os("TOOLPATH_CONFIG_DIR").unwrap()) + .join("documents"); + let cached: Vec = std::fs::read_dir(&documents) + .unwrap() + .map(|e| e.unwrap().file_name().to_string_lossy().into_owned()) + .collect(); + assert_eq!(cached.len(), 1, "expected one cached doc, got {cached:?}"); + assert!( + cached[0].ends_with("claude-abc.json"), + "unexpected cache id: {cached:?}" + ); +} + +#[test] +fn object_store_input_with_no_cache_leaves_the_cache_empty() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let graph = toolpath::v1::Graph::from_path(make_convo_path( + "agent:claude-code", + "claude-code://resume-s3-nocache", + )); + let uri = seed_object(bucket.path(), "claude-nocache", &graph); + + let recorder = RecordingExec::default(); + run_with_strategy( + ResumeArgs { + input: uri, + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: true, + force: false, + url: None, + }, + &recorder, + ) + .unwrap(); + + assert_eq!(recorder.captured().binary, "claude"); + let documents = std::path::PathBuf::from(std::env::var_os("TOOLPATH_CONFIG_DIR").unwrap()) + .join("documents"); + assert!( + !documents.exists() || std::fs::read_dir(&documents).unwrap().next().is_none(), + "--no-cache must not write the cache" + ); +} + +#[test] +fn object_store_input_prefers_the_cache_over_a_refetch() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let graph = toolpath::v1::Graph::from_path(make_convo_path( + "agent:claude-code", + "claude-code://resume-s3-cached", + )); + let uri = seed_object(bucket.path(), "claude-cached", &graph); + + let args = || ResumeArgs { + input: uri.clone(), + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: false, + force: false, + url: None, + }; + run_with_strategy(args(), &RecordingExec::default()).unwrap(); + + // Delete the object: a cache hit must not need it any more. + std::fs::remove_file(bucket.path().join("claude-cached.json")).unwrap(); + let recorder = RecordingExec::default(); + run_with_strategy(args(), &recorder).unwrap(); + assert_eq!(recorder.captured().binary, "claude"); +} + +#[test] +fn missing_object_reports_not_found() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let err = run_with_strategy( + ResumeArgs { + input: format!("file://{}/absent.json", bucket.path().display()), + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: false, + force: false, + url: None, + }, + &RecordingExec::default(), + ) + .unwrap_err(); + assert!(err.to_string().contains("not found"), "actual: {err}"); +} + +// ── Resuming from a destination rather than a document ────────────── +// +// `path share` has a picker across every harness; resuming from where +// you shared to needs one too, or a destination is a write-only hole. + +#[test] +fn a_destination_holding_one_document_resumes_it_without_asking() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let graph = toolpath::v1::Graph::from_path(make_convo_path( + "agent:claude-code", + "claude-code://resume-only-one", + )); + seed_object(bucket.path(), "2026-08-07-only-one-claude-abc", &graph); + + let recorder = RecordingExec::default(); + run_with_strategy( + ResumeArgs { + // The destination, not a document inside it. + input: bucket.path().to_string_lossy().into_owned(), + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: false, + force: false, + url: None, + }, + &recorder, + ) + .unwrap(); + + assert_eq!(recorder.captured().binary, "claude"); +} + +#[test] +fn an_empty_destination_says_how_to_fill_it() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let err = run_with_strategy( + ResumeArgs { + input: bucket.path().to_string_lossy().into_owned(), + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: false, + force: false, + url: None, + }, + &RecordingExec::default(), + ) + .unwrap_err(); + let s = err.to_string(); + assert!(s.contains("no shared documents"), "actual: {s}"); + assert!(s.contains("path p export object --to"), "actual: {s}"); +} + +#[test] +fn a_destination_url_without_a_json_suffix_is_browsed_not_fetched() { + let _env = env_lock(); + let _home = ScopedHome::new(); + let _path = ScopedPath::with_binary("claude"); + let cwd = tempfile::tempdir().unwrap(); + let bucket = tempfile::tempdir().unwrap(); + + let graph = toolpath::v1::Graph::from_path(make_convo_path( + "agent:claude-code", + "claude-code://resume-url-browse", + )); + seed_object(bucket.path(), "2026-08-07-browse-claude-abc", &graph); + + let recorder = RecordingExec::default(); + run_with_strategy( + ResumeArgs { + // A `file://` URL naming the container, not an object. + input: format!("file://{}", bucket.path().display()), + cwd: Some(cwd.path().to_path_buf()), + harness: Some(Harness::Claude), + no_cache: false, + force: false, + url: None, + }, + &recorder, + ) + .unwrap(); + + assert_eq!(recorder.captured().binary, "claude"); +} + // ── Rejection cases ───────────────────────────────────────────────── #[test] diff --git a/crates/toolpath-cli/Cargo.toml b/crates/toolpath-cli/Cargo.toml index a3aa6782..31b0dd21 100644 --- a/crates/toolpath-cli/Cargo.toml +++ b/crates/toolpath-cli/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "toolpath-cli" -version = "0.18.0" +version = "0.19.0" edition = "2024" license = "Apache-2.0" repository = "https://github.com/empathic/toolpath" @@ -14,7 +14,7 @@ name = "path" path = "src/main.rs" [dependencies] -path-cli = { path = "../path-cli", version = "0.18.0" } +path-cli = { path = "../path-cli", version = "0.19.0" } anyhow = "1.0" [workspace] diff --git a/site/_data/crates.json b/site/_data/crates.json index da0fcd41..0d6bed32 100644 --- a/site/_data/crates.json +++ b/site/_data/crates.json @@ -113,7 +113,7 @@ }, { "name": "path-cli", - "version": "0.18.0", + "version": "0.19.0", "description": "Unified CLI (binary: path)", "docs": "https://docs.rs/path-cli", "crate": "https://crates.io/crates/path-cli", @@ -121,7 +121,7 @@ }, { "name": "toolpath-cli", - "version": "0.18.0", + "version": "0.19.0", "description": "Deprecated alias for path-cli", "docs": "https://docs.rs/toolpath-cli", "crate": "https://crates.io/crates/toolpath-cli",