Define a Rust ABI

[steveklabnik]

Right now, Rust has no defined ABI. That may or may not be something we want eventually.

[ranma42]

CC me

[nrc]

See #1675 for some motivation for this feature (implementing plugins, that is plugins for Rust programs, not for the compiler).

[genodeftest]

Another motivation is the ability to ship shared libraries which could be reused by multiple applications on disk (reducing bandwith usage on update, reducing disk usage) and in RAM (through shared .text pages, reducing RAM usage).

[sdroege]

It would also make Linux distributions more happy as it would allow usage of shared libraries. Which apart from memory reduction in different ways, also make handling of security issues (or otherwise important bugs) simpler. You only have to update the code in a single place and rebuild that, instead of having to fix lots of copies of the code in different versions and rebuild everything.

[Arzte]

Shared libraries can be cool provided that there is a way to work around similar libraries accomplishing something in different ways such as openssl & libressl. There could also be value with making a way to allow a switch in-between static & dynamic libraries that can be set by the person compiling the crate. (possible flag?)

[Conan-Kudo]

In my opinion, it's very hard to take Rust seriously as a replacement systems programming language if it's not possible in any reasonably sane manner to build applications linking to libraries with a safe, stable ABI.

There are a lot of good reasons for supporting shared libraries, not the least of which is building fully-featured systems for more resource constrained devices (like your average SBC or mobile device). Not having shared libraries really blows up the disk utilization in places where it's not cheap.

@jpakkane describes this really well in his blog post where he conducts an experiment to prove this problem.

[vks]

In my opinion, it's very hard to take Rust seriously as a replacement systems programming language if it's not possible in any reasonably sane manner to build applications linking to libraries with a safe, stable ABI.

Note that C++ still doesn't have a stable ABI, and it took C decades to get one.

[steveklabnik]

It would also make Linux distributions more happy as it would allow usage of shared libraries.

There are a lot of good reasons for supporting shared libraries

(two quotes from two different people) Note that rust does support shared libraries. What it doesn't support is mixing them from different compiler toolchains. Some linux distros already do this with Rust; since they have one global rustc version, it works just fine.

[nagisa]

Rust also supports exporting functions with stable ABI just fine, as, for example, this shows.

[sdroege]

Note that C++ still doesn't have a stable ABI, and it took C decades to get one.

While that is true, implementations (GCC, clang, MSVC at least) have a (somewhat) defined ABI and it only changes every now and then. With Rust there is no defined ABI at all and things might break in incompatible ways with any change in the compiler, a library you're using or your code, and you can't know when this is the case as the ABI is in no way defined (sure, you could look at the compiler code, but that could also change any moment).

What it doesn't support is mixing them from different compiler toolchains. Some linux distros already do this with Rust; since they have one global rustc version, it works just fine.

The problem is not only about compiler versions, but as written above about knowing what you can change in your code without breaking your ABI. And crates generally tracking their ABI in one way or another. Otherwise you can't use shared libraries created from crates in any reasonable way other than recompiling everything (and assuming ABI changed) whenever something changes.

[cuviper]

At least on Linux, everyone has pretty much settled on the Itanium C++ ABI. But even with the compiler locked down, it still requires very careful maintenance by a library author who hopes to export a stable ABI. Check out these KDE policies, for instance.

Rust crates would have many of the same challenges in presenting a stable ABI. Plus I think this is actually compounded by not having the separation between headers and source, so it's harder to actually tell what is reachable code. It's much more than just pub -- any generic code that gets monomorphized in your consumers may have many layers of both public and private calls to make to the original crate's library. And all of that monomorphized code has to remain supported as-is when you're shipping updates to your crate.

[plietar]

Had a long term crazy idea to solve this.

Define a stable format for MIR and distribute all Rust binaries/shared libraries in that format. Package managers have a post-install step to translate the MIR into executables or .so files. Only the version of the MIR->binary backend "linker" has to be the same, the version of the source->MIR frontend compiler can differ.

Because of monomorphization and unboxed types, you need to still need to relink all the MIR files when a dependency is updated. Similarily, updating the backend compiler requires all the MIR files to be recompiled.

However, assuming we can push more and more optimisation passes into MIR rather than llvm, the time spent in the backend should be reduced to something acceptable.

If you want to push it even further, keep everything in MIR form and use miri (or a JIT version of it) to run them. Frequently used files can be linked and persisted to disk. And we've just reinvented the JVM/CLR/Webassembly.