Save/load incremental compilation dep graph
Contains the code to serialize/deserialize the dep graph to disk between executions. We also hash the item contents and compare to the new hashes. Also includes a unit test harness. There are definitely some known limitations, such as https://github.com/rust-lang/rust/issues/32014 and https://github.com/rust-lang/rust/issues/32015, but I am leaving those for follow-up work.
Note that this PR builds on https://github.com/rust-lang/rust/pull/32007, so the overlapping commits can be excluded from review.
r? @michaelwoerister
We used to track, for each crate, a path that led to the extern-crate
that imported it. Instead of that, track the def-id of the extern crate,
along with a bit more information, and derive the path on the fly.
We want to prevent compiling something against one version
of a dynamic library and then, at runtime accidentally
using a different version of the dynamic library. With the
old symbol-naming scheme this could not happen because every
symbol had the SVH in it and you'd get an error by the
dynamic linker when using the wrong version of a dylib. With
the new naming scheme this isn't the case any more, so this
patch adds the "link-guard" to prevent this error case.
This is implemented as follows:
- In every crate that we compile, we emit a function called
"__rustc_link_guard_<crate-name>_<crate-svh>"
- The body of this function contains calls to the
"__rustc_link_guard" functions of all dependencies.
- An executable contains a call to it's own
"__rustc_link_guard" function.
As a consequence the "__rustc_link_guard" function call graph
mirrors the crate graph and the dynamic linker will fail if a
wrong dylib is loaded somewhere because its
"__rustc_link_guard" function will contain a different SVH in
its name.
Automated conversion using the untry tool [1] and the following command:
```
$ find -name '*.rs' -type f | xargs untry
```
at the root of the Rust repo.
[1]: https://github.com/japaric/untry
- Rewrites the overlap checker to instead build up a specialization
graph, checking for overlap errors in the process.
- Use the specialization order during impl selection.
This commit does not yet handle associated types correctly, and assumes
that all items are `default` and are overridden.
Distinguish fn item types to allow reification from nothing to fn pointers.
The first commit is a rebase of #26284, except for files that have moved since.
This is a [breaking-change], due to:
* each FFI function has a distinct type, like all other functions currently do
* all generic parameters on functions are recorded in their item types, e.g.:
`size_of::<u8>` & `size_of::<i8>`'s types differ despite their identical signature.
* function items are zero-sized, which will stop transmutes from working on them
The first two cases are handled in most cases with the new coerce-unify logic,
which will combine incompatible function item types into function pointers,
at the outer-most level of if-else chains, match arms and array literals.
The last case is specially handled during type-checking such that transmutes
from a function item type to a pointer or integer type will continue to work for
another release cycle, but are being linted against. To get rid of warnings and
ensure your code will continue to compile, cast to a pointer before transmuting.
There's a lot of stuff wrong with the representation of these types:
TyFnDef doesn't actually uniquely identify a function, TyFnPtr is used to
represent method calls, TyFnDef in the sub-expression of a cast isn't
correctly reified, and probably some other stuff I haven't discovered yet.
Splitting them seems like the right first step, though.
The purpose of the translation item collector is to find all monomorphic instances of functions, methods and statics that need to be translated into LLVM IR in order to compile the current crate.
So far these instances have been discovered lazily during the trans path. For incremental compilation we want to know the set of these instances in advance, and that is what the trans::collect module provides.
In the future, incremental and regular translation will be driven by the collector implemented here.
This PR changes the `emit_opaque` and `read_opaque` methods in the RBML library to use a space-efficient binary encoder that does not emit any tags and uses the LEB128 variable-length integer format for all numbers it emits.
The space savings are nice, albeit a bit underwhelming, especially for dynamic libraries where metadata is already compressed.
| RLIBs | NEW | OLD |
|--------------|--------|-----------|
|libstd | 8.8 MB | 10.5 MB |
|libcore |15.6 MB | 19.7 MB |
|libcollections| 3.7 MB | 4.8 MB |
|librustc |34.0 MB | 37.8 MB |
|libsyntax |28.3 MB | 32.1 MB |
| SOs | NEW | OLD |
|---------------|-----------|--------|
| libstd | 4.8 MB | 5.1 MB |
| librustc | 8.6 MB | 9.2 MB |
| libsyntax | 7.8 MB | 8.4 MB |
At least this should make up for the size increase caused recently by also storing MIR in crate metadata.
Can this be a breaking change for anyone?
cc @rust-lang/compiler
