Nightly sync mode by flip1995 · Pull Request #58 · rust-lang/josh-sync

flip1995 · 2026-09-08T11:44:09Z

Clippy wants to pin its version in the repo to a nightly, so that contributors don't need any special tooling in order to compile it and rustup will automatically take care of toolchain management.

This requires some extra logic for syncing:

read and modify the rust-toolchain.toml file -> toml_edit
get the latest nightly -> ureq -> date+upstream_sha -> nightly-{date}

This PR moves a bunch of code around to add the new nightly sync-mode as just a new helper function. More details of the separate steps in the commit messages.

WIP, as I'm still testing it with this config:

org = "rust-lang"
repo = "rust-clippy"
filter = ":~(history=\"keep-trivial-merges,no-splice\")[:rev(<=7c06e7c42a8b56f9c1ac20e9ce34cadca0fce100:prefix=src/tools/clippy,<=da5114692c9ebe46b869488c5f34f92eb10b98c1:SQUASH)]:/src/tools/clippy"
filter-version = 2
pull-mode = "Nightly"
post-pull = [ { cmd = [ "cargo dev sync update_nightly" ], commit-message = "Update nightly in clippy-utils README.md"} ]

Kobzol · 2026-09-08T12:20:57Z

Hi, thanks for the PR. Fiddling with the TOML adds some complexity, but it makes sense.

I wonder, instead of using dates manually, maybe we can download https://static.rust-lang.org/manifests.txt and read the latest entry?

bjorn3 · 2026-09-10T08:25:18Z

manifests.txt in the past on occasion not been getting updated for several days in a row.

Kobzol · 2026-09-10T08:33:01Z

I think that only happens if we forget to update the CI job which needs to be poked once every 6 months I think 😆 But I realized that adding a dependency on a network client is perhaps a bit of a heavy hammer for this functionality.

Using the local nightly rustc to figure out its commit hash actually seems quite elegant. It could even work on CI, if we do something like rustup update nightly, and then read both the date and the commit hash from it. That might be the simplest solution, after all.

This PR already reads the hash from the nightly rustc, so maybe it could also read its date from it? Or you could pass the nightly verison explicitly. It seems better to me than guessing the date, because the nightly dates are always offset-by-one and it gets confusing.

RalfJung · 2026-09-10T08:41:59Z

base-commit = "Nightly"
rust-version-path = "rust-toolchain.toml"

These only really make sense in combination, right? IMO they should be controlled by a single flag.

bjorn3 · 2026-09-10T08:42:34Z

It could even work on CI, if we do something like rustup update nightly, and then read both the date and the commit hash from it.

That would mean you did have to download rustc twice, right? Once for the nightly toolchain and once for nightly-$(date). I did rather either fetch both date and commit from the nightly manifest: https://static.rust-lang.org/dist/channel-rust-nightly.toml or take the current date and then download the corresponding nightly toolchain and get the commit from it.

It seems better to me than guessing the date, because the nightly dates are always offset-by-one and it gets confusing.

The date reported by rustc is off by one, but the date in the toolchain name matches the current day.

Kobzol · 2026-09-10T08:47:11Z

Ok, fair enough. Since we would anyway be dealing with parsing TOML files, downloading from the manifest seems reasonable, as it is the source of truth for Rustup too, and contains both the date and the commit.

flip1995 · 2026-09-13T14:49:51Z

Thanks for the feedback! I swapped out chrono with ureq and now get the latest nightly + upstream_sha from the manifest file.

I also moved the rust_version_path config back to the shared CLI args. As Ralf pointed out this wasn't really necessary and we can just default to rust-toolchain.toml in the Nightly mode.

chrono is used to get the current date in order to update the nightly version, in case josh-sync is configured to use nightly versions. toml_edit is used to read and modify the rust-toolchain.toml file.

In the `Latest` sync mode, this will be `rust-version`, but in the `Nightly` mode this either has to be `rust-toolchain` (legacy) or `rust-toolchain.toml`.

This lets repos choose if they want to sync from the latest rustc commit or if they want to pin a nightly in rust-toolchain.toml.

Gets the current rust version depending on the BaseCommit mode selected.

Extracts the rust-version write logic out of the `rustc_pull` function in its own `bump_version_latest` helper. Then it adds a new nightly sync helper called `bump_version_nightly`. In the new mode, it uses toml_edit to modify the `rust-toolchain.toml` file with the latest nightly version, which it constructs through the current date. Small behaviorial change: The `new upstream base: <sha>` message is now printed after the `original local HEAD: <sha>` message.

It's default value is implied by the `base-commit` in the config file. No need to require configuring this and the `base-commit` separately.

This drops the chrono dependency and uses the nightly version from the latest nightly manifest instead, by pulling it with ureq.

flip1995 · 2026-09-13T14:51:33Z

Rebased onto main to drop the josh version bump commit.

Kobzol

Thank you! Left a few nits.

Kobzol · 2026-09-14T18:00:05Z

Thanks for making the changes! After the Result<Option<...>> change, I will also try this locally, otherwise it looks good to me.

… parsed Before it just printed an error and returned. If the file doesn't exist, it will return `Ok(None)`, indicating that no prior pull has happened.

Kobzol

Looks good, thank you! I tested it with rust-version and it seems to work fine. I didn't test the rust-toolchain.toml version, but I'm sure that if you'll run into any issues during the Clippy migration, we can flesh out the fixes then.

Thank you very much for implementing this!

flip1995 · 2026-09-15T13:38:54Z

but I'm sure that if you'll run into any issues during the Clippy migration, we can flesh out the fixes then.

Yes, I have the Clippy PR open here: rust-lang/rust-clippy#17739. I'm doing another subtree sync on Thursday and will then use your script for the roundtrip test on Friday (or on the weekend).

If anything doesn't work out, I'll open another PR fixing that. But in my initial tests everything worked out.

Side note: one feature I'd like to still add is a rustc-josh-sync pull --continue flag that can be used if there were merge conflicts during the pull. Otherwise it is easy to forget to run post-pull commands or opening the PR with the correct message.

Kobzol · 2026-09-15T13:41:44Z

That sounds great. Here is an updated/fixed version of my script: https://gist.github.com/Kobzol/7c8d2a5059e5e2483c7807f0007e25cd

flip1995 marked this pull request as ready for review September 13, 2026 14:48

flip1995 added 7 commits September 13, 2026 16:51

Add toml_edit and chrono dependencies

5d1d913

chrono is used to get the current date in order to update the nightly version, in case josh-sync is configured to use nightly versions. toml_edit is used to read and modify the rust-toolchain.toml file.

Move rustc_version_path to the config file

3d6ab20

In the `Latest` sync mode, this will be `rust-version`, but in the `Nightly` mode this either has to be `rust-toolchain` (legacy) or `rust-toolchain.toml`.

Add BaseCommit config option

533e3c5

This lets repos choose if they want to sync from the latest rustc commit or if they want to pin a nightly in rust-toolchain.toml.

Introduce rust_version helper

a45e572

Gets the current rust version depending on the BaseCommit mode selected.

Move rust_version_path back to the shared CLI args

b65748f

It's default value is implied by the `base-commit` in the config file. No need to require configuring this and the `base-commit` separately.

Get the nightly version+commit from the manifest

807067d

This drops the chrono dependency and uses the nightly version from the latest nightly manifest instead, by pulling it with ureq.

flip1995 force-pushed the nightly-sync-mode branch from e63a224 to 807067d Compare September 13, 2026 14:51

Kobzol reviewed Sep 14, 2026

View reviewed changes

Comment thread src/sync.rs Outdated

Comment thread src/sync.rs Outdated

Comment thread src/sync.rs Outdated

Comment thread src/bin/rustc_josh_sync.rs Outdated

Rename base-commit -> pull-mode and other clean ups

005d970

flip1995 requested a review from Kobzol September 14, 2026 14:55

flip1995 force-pushed the nightly-sync-mode branch from b941484 to 8a5d5a0 Compare September 14, 2026 16:18

Make last_pulled_upstream_sha return an error if the version can't be…

29d320f

… parsed Before it just printed an error and returned. If the file doesn't exist, it will return `Ok(None)`, indicating that no prior pull has happened.

flip1995 force-pushed the nightly-sync-mode branch from 8a5d5a0 to 29d320f Compare September 15, 2026 09:37

Kobzol approved these changes Sep 15, 2026

View reviewed changes

Kobzol merged commit d4a761a into rust-lang:main Sep 15, 2026
2 checks passed

flip1995 deleted the nightly-sync-mode branch September 15, 2026 13:38

Conversation

flip1995 commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kobzol commented Sep 8, 2026

Uh oh!

bjorn3 commented Sep 10, 2026

Uh oh!

Kobzol commented Sep 10, 2026

Uh oh!

RalfJung commented Sep 10, 2026

Uh oh!

bjorn3 commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kobzol commented Sep 10, 2026

Uh oh!

flip1995 commented Sep 13, 2026

Uh oh!

flip1995 commented Sep 13, 2026

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kobzol commented Sep 14, 2026

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

flip1995 commented Sep 15, 2026

Uh oh!

Kobzol commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

flip1995 commented Sep 8, 2026 •

edited

Loading

bjorn3 commented Sep 10, 2026 •

edited

Loading