Use a relative bindir for rustdoc to find rustc
In bootstrap, we set `RUSTC_INSTALL_BINDIR` to `config.bindir`, so
rustdoc can find rustc relative to the toolchain sysroot. However, if a
distro script like Fedora's `%configure` sets an absolute path, then
rustdoc's `sysroot.join(bin_path)` ignores that sysroot altogether.
That would be OK once the toolchain is actually installed, but it breaks
the in-tree doc tests during the build, since `/usr/bin/rustc` is still
the old version. So now we try to make `RUSTC_INSTALL_BINDIR` relative
to the sysroot prefix in the first place.
r? @Mark-Simulacrum
In bootstrap, we set `RUSTC_INSTALL_BINDIR` to `config.bindir`, so
rustdoc can find rustc relative to the toolchain sysroot. However, if a
distro script like Fedora's `%configure` sets an absolute path, then
rustdoc's `sysroot.join(bin_path)` ignores that sysroot altogether.
That would be OK once the toolchain is actually installed, but it breaks
the in-tree doc tests during the build, since `/usr/bin/rustc` is still
the old version. So now we try to make `RUSTC_INSTALL_BINDIR` relative
to the sysroot prefix in the first place.
Split the rustc target libraries into separate rustc-dev component
This is re-applies a squashed version of #64823 as well as including #65337 to fix bugs noted after merging the first PR.
The second PR is confirmed as fixing windows-gnu, and presumably also fixes other platforms, such as musl (i.e. #65335 should be fixed); `RUSTUP_DIST_SERVER=https://dev-static.rust-lang.org rustup toolchain install nightly-2019-10-16` can be installed to confirm that this is indeed the case.
- Compatible with Emscripten 1.38.46-upstream or later upstream.
- Refactors the Emscripten target spec to share code with other wasm
targets.
- Replaces the old incorrect wasm32 C call ABI with the correct one,
preserving the old one as wasm32_bindgen_compat for wasm-bindgen
compatibility.
- Updates the varargs ABI used by Emscripten and deletes the old one.
- Removes the obsolete wasm32-experimental-emscripten target.
- Uses EMCC_CFLAGS on CI to avoid the timeout problems with #63649.
This splits out a rustc-dev component with the compiler crates, and
keeps the status quo of default installed files on nightly. The default
changing to not install compiler libraries by default is left for a
future pull request.
However, on stable and beta, this does remove the compiler libraries
from the set of libraries installed by default, as they are never needed
there (per our stability story, they "cannot" be used).
minimize the rust-std component
This changes the `rust-std` dist component to only include the artifacts of compiling the `libstd` step, as listed in `.libstd.stamp`. This does include `test` and `proc-macro` as well. The remaining _unstable_ libraries that are built as part of `rustc` are packaged into a new `rustc-dev` component, intended for use in the development of closely related tools (clippy, miri, rls).
Here are the component sizes from the [try build](https://dev-static.rust-lang.org/dist/2019-10-07/index.html):
| Name | Size
| --- | ---
| rust-std-nightly-x86_64-unknown-linux-gnu.tar.gz | 23.94 MiB
| rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | 17.4 MiB
| rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.gz | 182.03 MiB
| rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | 157.91 MiB
Fixes#61978Fixes#62486
This is needed to ensure that the crates during a normal build are
shared with the crates during testing, otherwise they'll end up hasing
differently and we'll recompile crates like `core` during tests.
This commit changes the return type of `Builder::cargo` to return a
builder that allows dynamically adding more `RUSTFLAGS` values
after-the-fact. While not used yet, this will later be used to delete
more of `rustc.rs`
Most of `bootstrap/bin/rustc.rs` doesn't need to exist with the advent
of `RUSTFLAGS` (yes this is super old) so this starts by refactoring a
bit to make it easier locally in the `Builder::cargo` method to append
to `RUSTFLAGS` that gets down to rustc.
This ensures that the failure cases for finding the codegen backend and
for finding the rustc binary are essentially the same, and since we
almost always will load the codegen backend, this is essentially meaning
that the rustc change is not a regression.
Since its inception rustbuild has always worked in three stages: one for
libstd, one for libtest, and one for rustc. These three stages were
architected around crates.io dependencies, where rustc wants to depend
on crates.io crates but said crates don't explicitly depend on libstd,
requiring a sysroot assembly step in the middle. This same logic was
applied for libtest where libtest wants to depend on crates.io crates
(`getopts`) but `getopts` didn't say that it depended on std, so it
needed `std` built ahead of time.
Lots of time has passed since the inception of rustbuild, however,
and we've since gotten to the point where even `std` itself is depending
on crates.io crates (albeit with some wonky configuration). This
commit applies the same logic to the two dependencies that the `test`
crate pulls in from crates.io, `getopts` and `unicode-width`. Over the
many years since rustbuild's inception `unicode-width` was the only
dependency picked up by the `test` crate, so the extra configuration
necessary to get crates building in this crate graph is unlikely to be
too much of a burden on developers.
After this patch it means that there are now only two build phasese of
rustbuild, one for libstd and one for rustc. The libtest/libproc_macro
build phase is all lumped into one now with `std`.
This was originally motivated by rust-lang/cargo#7216 where Cargo was
having to deal with synthesizing dependency edges but this commit makes
them explicit in this repository.
This patch supports less behavior than before, since specifiying stage 1
vs stage 2 is no longer possible, but that is presumably a somewhat rare
use case anyway, so not supporting it seems acceptable (and it can be
readded easily if desired).
Utilize -Zbinary-dep-depinfo in rustbuild
We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.
Both of these are fixable in the longer term but this existing patch gives us the following benefits:
* We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes#54712.
* Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes#54008, fixes#50481.
* We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes#53284, and fixes#52719.
* Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes#49979 (but no reproduction steps in that issue). Fixes#59105.
cc #63012
