WIP Add Nix Flake devshell + direnv#432
Conversation
|
|
||
| channel = "1.83.0" # check ^^^ when changing this | ||
| components = ["rust-analyzer"] | ||
| components = ["rust-analyzer", "llvm-tools-preview"] |
There was a problem hiding this comment.
Support host stripping performed in scripts/build.py
| def rust_target_is_installed(rust_target: str) -> bool: | ||
| result = subprocess.run( | ||
| ["rustc", "--print", "target-libdir", "--target", rust_target], | ||
| check=False, | ||
| stdout=subprocess.PIPE, | ||
| stderr=subprocess.DEVNULL, | ||
| text=True, | ||
| ) | ||
| return result.returncode == 0 and Path(result.stdout.strip()).is_dir() | ||
|
|
||
|
|
||
| def install_rust_target(rust_target: str, *, required: bool = False) -> None: | ||
| if rust_target_is_installed(rust_target): | ||
| return | ||
|
|
||
| if shutil.which("rustup") is None: | ||
| message = ( | ||
| f"Rust target {rust_target} is not installed and rustup is unavailable; " | ||
| "enter `nix develop` or install the target manually" | ||
| ) | ||
| if required: | ||
| raise SystemExit(message) | ||
| print(f"warning: {message}") | ||
| return | ||
|
|
||
| run("rustup", "target", "add", rust_target, check=required) |
There was a problem hiding this comment.
Previously we assumed Rust targets were always managed bu rustup and unconditionally ran this every time, even if the target was already installed. Nix manages this via the nix store and does not use this rustup. So this would fail with rustup: command not found even if all the targets were available. By using rustc to ask where the target libraries should be, then verifying it exists there via result.returncode == 0 and Path(result.stdout.strip()).is_dir(), we maintain backward compat for non-nix users.
f6a25cf to
09001ed
Compare
Restores and updates the old flake.nix to add a post-Zig migration devshell. Also adds direnv for automatic env loading upon entering the directory. Since Zig compiler has no stable release yet, this uses roc-overlay to always use latest nightly roc compiler, so the user doesn't have to continuously download nightly. Adds checks to scripts/build.py to not download the rust toolchain on every build if it already exists, and to allow interop with Nix users and backward compatibility for non Nix users. Also updates CONTRIBUTING.md with new Nix usage section. docs: note required Nix features
c9088af to
b6f7f5b
Compare
|
@Anton-4 can you please review this when you get a chance. |
Anton-4
left a comment
Anton-4
left a comment
There was a problem hiding this comment.
Thanks for setting this up @thebrandonlucas :)
It looks pretty good based on a quick first look.
To solve that here we'd have to pin a particular nightly release and just keep basic-cli up to date with it, modifying the flake with every incompatibility.
I was thinking that this could actually work out well in practice, given the amount of breaking changes we're currently doing. It's actually quite nice for Roc users to know which nightly is fully compatible and tested with basic-cli. What do you think?
@Anton-4 Probably depends on priorities. On one hand we would like to just stay up to date with the latest compiler (in which case we'd use roc-overlay). This is the current solution I opted for because it keeps us constantly up to date and is less dev overhead for anyone working on Pros:
Cons:
If the rate of breaking changes in nightly outweighs these benefits or stunts I'd favor using the roc-overlay for now until we get to stable personally, but I'm not up to speed on where the priorities lie so you just let me know if you want me to change anything :) |
|
Thanks for the thorough reply @thebrandonlucas! I feel like stability and reliability is Roc's biggest weakness right now so that is what I would like to prioritize. basic-cli is our most important platform and a cornerstone for the Roc brand, so I still want to opt for pinning the nightly. We also talked about implementing a feature that pins the Roc version in the main.roc file so in the future we would need to pin it in nix too anyway. I am sure roc-overlay will be appreciated and used by the Roc community though! |
Anton-4
changed the title
|
#444 sounds good :) |
|
@Anton-4 roc-overlay can still be used to simplify this because it keeps track of all nightly compilers. So whether we pin or use latest it's still useful ;) |
|
This will sound harsh but we always want to minimize dependencies and supply chain risks so I would like to avoid adding a third party controlled project like roc-overlay to basic-cli. |
No worries, I understand. Just wanted to point out that it could be used both ways to account for flexible use cases. I can update the PR to just be a self-contained flake pinned to a specific nightly if that's the direction we want to go, just need to know which one to use. |
|
Thanks for your understanding @thebrandonlucas :) This nightly was used for the latest PR so all tests should pass with it. |
|
@Anton-4 I made a commit locally to update this PR to pin an exact version but before I push I wanted to ask: If the only reason not to use roc-overlay is to reduce 3rd party supply chain risk, I'm more than happy to hand it off to the Roc org so that it can be reused by this and other projects. It does/would simplify repeated work that I think could be reusable both for dev shells and CI (for ex. this ticket) for this and other projects. I don't want to bikeshed though, so if not just give me the word and I'll go ahead and push the standalone flake. |
That would be a great solution @thebrandonlucas! |
|
Awesome, just let me know when/who to transfer to and I'll make it happen! |
|
You can transfer it to the roc-lang org: https://github.com/roc-lang |
3 participants
flake.nixwas removed from #413 to avoid bundling another change into that large PR.I've added back the flake.nix as mentioned here removing unnecessary tools and adding new ones (Zig 16, etc.). One problem I ran into was that while the old
flake.nixwas able to just use the standard Roc compiler from unstable nixpkgs, that doesn't work for the new compiler because it's unreleased on nixpkgs. To solve that here we'd have to pin a particular nightly release and just keepbasic-cliup to date with it, modifying the flake with every incompatibility.That seemed like too much overhead to be worth it, and I felt that this must be a general pain point for other projects, so I created roc-overlay to solve this problem for this and other projects that wish to use it. It's modeled after zig-overlay and is useful for any project which intends to keep up with the latest nightly compilers (or pin themselves to previous ones). More here.
Updated
CONTRIBUTING.mdto add the section about using Nix for the development environment. We could probably also take advantage of this to simplify CI (and have a single source of truth) onceroc-overlayproves it's mettle a bit more.I also added direnv which is a great convenience tool for automatically loading dev environments upon shell entry. It requires manual activation by the user once, then subsequently loads the shell with all appropriate dependencies thereafter.