diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 537b2394..24d00b4f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -147,13 +147,13 @@ The pattern is: Releases are published in two places: GitHub releases, and `https://trunk.io/releases/analytics-cli/prod/` (the `trunk-releases` S3 bucket). Trunk's own consumers (the `analytics-uploader` action and the `trunk` CLI's `flakytests` command) read the S3 copy. GitHub releases stay, because the frozen legacy `trunk` launcher and customers' own scripts download from them. 1. **Cut**: run the `Release` workflow with the version. It builds, smoke-tests, creates a GitHub **prerelease**, and uploads the same assets to `releases/analytics-cli/prod//`. A version is immutable once published. -2. **Promote**: run the `Promote Release` workflow with the version. It points `channel-.json` at it. With **`promote-latest`** ticked (off by default), and when the version is the newest release overall, it also points `channel.json` at it and marks it latest on GitHub. Leave it off to hold `channel.json` and GitHub's latest back: analytics-uploader v2.1.7 and older follow one of them, and many orgs pin those versions. With it off, the `trunk` CLI's `flakytests` also keeps resolving `latest` to the previous release, unless the caller sets `TRUNK_ANALYTICS_CLI_VERSION`. Promoting a 0.x backport after 1.x moves only `channel-0.json` either way. A pointer never moves back unless you tick `allow-downgrade`, for a rollback. Do **not** mark a release latest in the GitHub UI, since that leaves the pointers behind. +2. **Promote**: run the `Promote Release` workflow with the version. It points `channel-.json` at it. With **`promote-latest`** ticked (off by default), and when the version is the newest release overall, it also points `channel.json` at it and marks it latest on GitHub. Leave it off to hold `channel.json` and GitHub's latest back: analytics-uploader v2.1.7 and older follow one of them, and many orgs pin those versions. The `trunk` CLI's `flakytests` (0.0.14 and later) reads `channel-1.json`, so it gets a 1.x release whether or not the box is ticked. Promoting a 0.x backport after 1.x moves only `channel-0.json` either way. A pointer never moves back unless you tick `allow-downgrade`, for a rollback. Do **not** mark a release latest in the GitHub UI, since that leaves the pointers behind. 3. **Backfill**: run `Mirror Releases to S3` to copy releases that predate the S3 lane (or that a failed run left incomplete). Pass space-separated tags or `all`. Re-running it is safe because complete versions are skipped. Each download is first checked against the sha256 digest GitHub recorded for it; releases from before June 2025 have no digest and are copied unchecked. Layout under `releases/analytics-cli/prod/`: -- `channel.json`: `{"latest": ""}`, the newest release overall. Read by the `trunk` CLI and the docs. -- `channel-.json`: the same shape, for the newest release of one major version (`channel-0.json`, `channel-1.json`). Each analytics-uploader major reads its own, so `@v2` stays on 0.x. +- `channel.json`: `{"latest": ""}`, the newest release overall. Read by the docs and by `trunk` CLI releases before 0.0.14. +- `channel-.json`: the same shape, for the newest release of one major version (`channel-0.json`, `channel-1.json`). Each analytics-uploader major reads its own, so `@v2` stays on 0.x. The `trunk` CLI reads `channel-1.json`, so that pointer must never be removed or pointed at another major. - Both are written with `Cache-Control: max-age=60` and are the only mutable objects. - `/`: each GitHub release asset under the same filename, e.g. `trunk-analytics-cli-x86_64-unknown-linux.tar.gz`. - `/manifest.json`: `{"version": "", "artifacts": {"": {"sha256": ""}}}`, uploaded last, so its presence means the version is complete. diff --git a/README.md b/README.md index fe0800bd..53af6568 100644 --- a/README.md +++ b/README.md @@ -20,16 +20,15 @@ This is downloaded automatically using the [Trunk launcher][launcher] and when u ## Credentials -`--token` / `TRUNK_API_TOKEN` takes one of two credentials. The CLI reads which one before it calls Trunk: +`--token` / `TRUNK_API_TOKEN` takes a **v2 API key** (`trunk_v2_...`). Create one at https://app.trunk.io/settings/developer/api-tokens, and give it the **Upload test results** (`upload_test_results`) permission. -- **A v2 API key** (`trunk_v2_...`). Create one at https://app.trunk.io/settings/developer/api-tokens, and give it the **Upload test results** (`upload_test_results`) permission. The CLI sends it as `Authorization: Bearer`. -- **A Trunk JWT**. The CLI sends it on `x-trunk-token`. With a key stored by `trunk auth api-key set`, `trunk flakytests upload` exchanges it for a JWT just before the upload and passes that in `TRUNK_API_TOKEN`, while `trunk flakytests test` passes the key itself. A JWT lasts at most 15 minutes, and the one `trunk flakytests upload` passes can have about a minute left, so use a JWT only for the upload itself: a long `trunk-analytics-cli test` run, or a test-runner plugin that looks up quarantined tests early and uploads at the end, can outlive it, so give those a v2 API key. When Trunk refuses an expired JWT, the CLI says so; it does not refresh it. +With the trunk CLI, store the key once with `trunk auth api-key set`, and `trunk flakytests` passes a credential for you. It may pass a short-lived Trunk token instead of the key; you don't need to manage it. -analytics-cli 1.0.0 refuses any other token, the organization token included, before it calls Trunk. Pin analytics-cli 0.17.x to keep using the organization token until December 1, 2027. +analytics-cli 1.0.0 refuses any other token, the organization token included, before it calls Trunk. Pin analytics-cli 0.17.x to keep using the organization token until December 1, 2027. To upgrade from 0.17.x, see [UPGRADING.md](UPGRADING.md). If Trunk answers `403` because the key lacks the Upload test results permission, edit the key on the API Keys screen to add it. -`--allow-forked-pr-uploads` uploads with no token, only from a GitHub Actions run of a pull request from a fork: the CLI swaps the job's `GITHUB_TOKEN` for a Trunk login. +`--allow-forked-pr-uploads` uploads with no token, only from a GitHub Actions run of a pull request from a fork. The job needs `GITHUB_TOKEN` in its environment. ## Development diff --git a/UPGRADING.md b/UPGRADING.md new file mode 100644 index 00000000..83ba6336 --- /dev/null +++ b/UPGRADING.md @@ -0,0 +1,61 @@ +# Upgrading from 0.17.x to 1.0.0 + +analytics-cli 1.0.0 uses only the Trunk API v2. It uploads only to test collections and takes only a v2 API key. Most runs need three changes: a v2 API key, a test collection id, and no removed flags. + +0.17.x stays available and keeps taking the organization token until December 1, 2027. Pin it while you move: it gets important fixes on the `0.17.x` branch, and no new features. + +## If you use analytics-uploader + +- **v3** installs the trunk CLI and runs `trunk flakytests`, which resolves analytics-cli 1.x. Move to it to get 1.0.0. +- **v2.1.8 and later**, and the floating `@v2` tag, stay on 0.17.x. +- **v2.1.7 and earlier, pinned by SHA or exact tag,** follow the latest analytics-cli release, which stays 0.17.x for now. 1.0.0 becomes the latest release in a later, separate step. From then on these versions get 1.0.0, and their uploads fail with an organization token, without `test-collection-id`, or with the `quarantine` or `hide-banner` input set, which they pass as flags 1.0.0 removed. Move to v2.1.8 or v3 before then, or set `cli-version: 0.17.0`. + +## If you use the trunk CLI + +From trunk CLI 0.0.14, `trunk flakytests` runs the newest analytics-cli 1.x unless `TRUNK_ANALYTICS_CLI_VERSION` pins a version. Earlier trunk CLI versions stay on 0.17.x until 1.0.0 becomes the latest release; run `trunk update` to get 0.0.14. + +## What you have to change + +### Credentials + +`--token` / `TRUNK_API_TOKEN` must hold a **v2 API key** (`trunk_v2_...`) with the **Upload test results** permission. Create it at https://app.trunk.io/settings/developer/api-tokens. With `trunk flakytests`, store the key once with `trunk auth api-key set` and the trunk CLI passes it for you. + +1.0.0 refuses the organization token before it calls Trunk; `test` refuses before it runs your tests. A key without Upload test results gets a `403` that names the permission. See [Credentials](README.md#credentials). + +### Test collection + +`--test-collection-id` / `TRUNK_TEST_COLLECTION_ID` is required for `upload` and `test`. A blank value counts as missing. 0.17.x inferred the collection from the repository; 1.0.0 does not. The rspec gem fails the run without one (see Test-runner plugins). + +### Removed flags and variables + +| Removed | Instead | +| ------------------------------------------------------------------- | --------------------------------------------------------- | +| `--use-quarantining` | Nothing for `true`; `--disable-quarantining` for `false`. | +| `--team`, `--print-files`, `--hide-banner` | Drop them; they did nothing. | +| `--public-repo-id` / `TRUNK_PUBLIC_REPO_ID` | `--allow-forked-pr-uploads` with `--test-collection-id`. | +| `--hide-test-collection-links` / `TRUNK_HIDE_TEST_COLLECTION_LINKS` | Drop it. Every link is now a test collection link. | + +`upload` and `validate` exit with code 2 on a removed flag. `test` reads an unknown flag before the test command as part of the command, so the run fails with exit code 70. + +### Fork pull requests + +`--allow-forked-pr-uploads` with no token works only in a GitHub Actions run of a pull request from a fork, and the job needs `GITHUB_TOKEN` in its environment. There is no fallback any more: Trunk refusing the job's token, a missing `GITHUB_TOKEN`, or the flag outside a fork pull request uploads nothing and says why. + +### Output + +- The summary goes to **stdout**; logs, notices and progress go to stderr. A script that reads only stderr loses the summary. +- `--output-format json` (or `TRUNK_OUTPUT_FORMAT=json`) on `upload` and `validate` prints the summary as the JSON that `--summary-output-file` writes. +- Test links point at the test in its test collection. The old repository-scoped test links are gone. + +### Network + +1.0.0 sends its telemetry to `api.trunk.io`, the host it already uploads to, instead of `telemetry.trunk.io`. A firewall allowlist needs only `api.trunk.io` and the storage host uploads go to. + +## Test-runner plugins + +The same rules apply to `test_report` and the rspec gem: a v2 API key and `TRUNK_TEST_COLLECTION_ID`. + +The rspec gem 1.0.0 **fails the run** when it is set up to upload but can't: with `TRUNK_ORG_URL_SLUG` and a token set, a missing `TRUNK_TEST_COLLECTION_ID` or an organization token raises `RSpec::Trunk::ConfigurationError` while `spec_helper` loads, so no examples run and RSpec exits non-zero. 0.17.x only warned and turned itself off. + +- To run without Trunk, set `DISABLE_RSPEC_TRUNK_FLAKY_TESTS=true`. An empty `TRUNK_API_TOKEN`, such as a fork pull request's missing secret, still turns the gem off quietly. +- The gem is published to RubyGems with each release, with no separate promotion step, so an unpinned `gem 'rspec_trunk_flaky_tests'` or a Dependabot or Renovate bump picks up 1.0.0 as soon as it is released. Pin `gem 'rspec_trunk_flaky_tests', '~> 0.17.0'` to stay on 0.17.x while you move. diff --git a/rspec-trunk-flaky-tests/README.md b/rspec-trunk-flaky-tests/README.md index 54d8621a..a3236af3 100644 --- a/rspec-trunk-flaky-tests/README.md +++ b/rspec-trunk-flaky-tests/README.md @@ -106,7 +106,7 @@ require 'trunk_spec_helper' ### Environment Variables -For a complete list of environment variables that the gem accepts, see [`lib/trunk_spec_helper.rb`](lib/trunk_spec_helper.rb). The gem uses the same environment variables as the Trunk Analytics CLI for configuration overrides. `TRUNK_ORG_URL_SLUG` and a non-empty `TRUNK_API_TOKEN` activate the plugin; without either it stays off. Once active it needs `TRUNK_TEST_COLLECTION_ID` too, and `TRUNK_API_TOKEN` must be a v2 API key with the Upload test results permission, or a Trunk JWT that outlasts the run (one lasts at most 15 minutes). With no test collection, or with an organization token, loading the plugin raises `RSpec::Trunk::ConfigurationError`, so RSpec runs no examples and exits non-zero; set `DISABLE_RSPEC_TRUNK_FLAKY_TESTS=true` to run without Trunk. See [Credentials](../README.md#credentials). +For a complete list of environment variables that the gem accepts, see [`lib/trunk_spec_helper.rb`](lib/trunk_spec_helper.rb). The gem uses the same environment variables as the Trunk Analytics CLI for configuration overrides. `TRUNK_ORG_URL_SLUG` and a non-empty `TRUNK_API_TOKEN` activate the plugin; without either it stays off. Once active it needs `TRUNK_TEST_COLLECTION_ID` too, and `TRUNK_API_TOKEN` must be a v2 API key with the Upload test results permission. With no test collection, or with an organization token, loading the plugin raises `RSpec::Trunk::ConfigurationError`, so RSpec runs no examples and exits non-zero; set `DISABLE_RSPEC_TRUNK_FLAKY_TESTS=true` to run without Trunk. See [Credentials](../README.md#credentials). #### `TRUNK_QUARANTINE_QUERY_FAILURE_EXIT`