This commit builds on #65501 continue to simplify the build system and
compiler now that we no longer have multiple LLVM backends to ship by
default. Here this switches the compiler back to what it once was long
long ago, which is linking LLVM directly to the compiler rather than
dynamically loading it at runtime. The `codegen-backends` directory of
the sysroot no longer exists and all relevant support in the build
system is removed. Note that `rustc` still supports a dynamically loaded
codegen backend as it did previously, it just no longer supports
dynamically loaded codegen backends in its own sysroot.
Additionally as part of this the `librustc_codegen_llvm` crate now once
again explicitly depends on all of its crates instead of implicitly
loading them through the sysroot. This involved filling out its
`Cargo.toml` and deleting all the now-unnecessary `extern crate`
annotations in the header of the crate. (this in turn required adding a
number of imports for names of macros too).
The end results of this change are:
* Rustbuild's build process for the compiler as all the "oh don't forget
the codegen backend" checks can be easily removed.
* Building `rustc_codegen_llvm` is much simpler since it's simply
another compiler crate.
* Managing the dependencies of `rustc_codegen_llvm` is much simpler since
it's "just another `Cargo.toml` to edit"
* The build process should be a smidge faster because there's more
parallelism in the main rustc build step rather than splitting
`librustc_codegen_llvm` out to its own step.
* The compiler is expected to be slightly faster by default because the
codegen backend does not need to be dynamically loaded.
* Disabling LLVM as part of rustbuild is still supported, supporting
multiple codegen backends is still supported, and dynamic loading of a
codegen backend is still supported.
This is not yet actually used by CI, but implements the logic for
checking that tools are properly building on beta/stable and during beta
cutoff week.
This attempts to mirror the checking functionality in
src/ci/docker/x86_64-gnu-tools/checktools.sh, and called scripts. It
does not attempt to run the relevant steps (that functionality was
originally desired to be moved into bootstrap as well, but doing so
proved more difficult than expected).
This is intended as a way to centralize and make clearer the logic
involved in toolstate checking. In particular, the previous logic was
spread across numerous python and shell scripts in such a way that made
interpretation quite difficult.
LLVM 7 is over a year old, which should be plenty for compatibility. The
last LLVM 6 holdout was llvm-emscripten, which went away in #65501.
I've also included a fix for LLVM 8 lacking `MemorySanitizerOptions`,
which was broken by #66522.
GitHub Actions: preparations, part 1
This PR adds the first batch of commits in preparation for GitHub Actions:
* Added GitHub Actions support in `src/ci/shared.sh` and bootstrap.
* Addded a `setup-environment.sh` script which guesses and sets the `DEPLOY`, `DEPLOY_ALT` and `IMAGE` environment variables automatically, to reduce the verbosity of the CI configuration.
This PR does **not** yet add any builders on GitHub Actions.
r? @alexcrichton
Use a relative bindir for rustdoc to find rustc
In bootstrap, we set `RUSTC_INSTALL_BINDIR` to `config.bindir`, so
rustdoc can find rustc relative to the toolchain sysroot. However, if a
distro script like Fedora's `%configure` sets an absolute path, then
rustdoc's `sysroot.join(bin_path)` ignores that sysroot altogether.
That would be OK once the toolchain is actually installed, but it breaks
the in-tree doc tests during the build, since `/usr/bin/rustc` is still
the old version. So now we try to make `RUSTC_INSTALL_BINDIR` relative
to the sysroot prefix in the first place.
r? @Mark-Simulacrum
make the error message more readable
When I type it wrong e.g. `--stage --bless`, missing a number here, this change would make it more explicit what's going wrong here.
In bootstrap, we set `RUSTC_INSTALL_BINDIR` to `config.bindir`, so
rustdoc can find rustc relative to the toolchain sysroot. However, if a
distro script like Fedora's `%configure` sets an absolute path, then
rustdoc's `sysroot.join(bin_path)` ignores that sysroot altogether.
That would be OK once the toolchain is actually installed, but it breaks
the in-tree doc tests during the build, since `/usr/bin/rustc` is still
the old version. So now we try to make `RUSTC_INSTALL_BINDIR` relative
to the sysroot prefix in the first place.
Upload toolstates.json to rust-lang-ci2
This PR does two things:
* Following up with https://github.com/rust-lang/rust/pull/65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended.
* Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot.
r? @alexcrichton
cc @RalfJung
Before this commit toolstates.json was stored in /tmp and it wasn't
mounted outside the build container. That caused uploading the file in
the upload-artifacts task to fail, as the file was missing on the host.
Mounting /tmp/toolstates.json alone is not the best approach: if the
file is missing when the container is started the Docker engine will
create a *directory* named /tmp/toolstates.json.
The Docker issue could be solved by pre-creating an empty file named
/tmp/toolstates.json, but doing that could cause problems if bootstrap
fails to generate the file and the toolstate scripts receive an empty
JSON.
The approach I took in this commit is to instead mount a /tmp/toolstate
directory inside Docker, and create the toolstates.json file in it. That
also required a small bootstrap change to ensure the directory is
created if it's missing.
Use rustc-workspace-hack for rustbook
As rustbook now depends transitively on openssl, it needs access to the
rustc-workspace-hack/all-static feature to pick up openssl-sys/vendored.
This fixes the rust build with `all-static = true` on systems where
openssl is not installed (e.g. when cross-compiling).
Re-enable Emscripten's exception handling support
Passes LLVM codegen and Emscripten link-time flags for exception
handling if and only if the panic strategy is `unwind`. Sets the
default panic strategy for Emscripten targets to `unwind`. Re-enables
tests that depend on unwinding support for Emscripten, including
`should_panic` tests.
r? @alexcrichton
As rustbook now depends transitively on openssl, it needs access to the
rustc-workspace-hack/all-static feature to pick up openssl-sys/vendored.
This fixes the rust build with `all-static = true` on systems where
openssl is not installed (e.g. when cross-compiling).
Redesign the interface to the unikernel HermitCore
We are developing the unikernel HermitCore, where the kernel is written in Rust and is already part of the Rust Standard Library. The interface between the standard library and the kernel based on a small C library. With this pull request, we remove completely the dependency to C and use lld as linker. Currently, the kernel will be linked to the application as static library, which is published at https://github.com/hermitcore/libhermit-rs.
We don’t longer support the C interface to the kernel. Consequently, we remove this part from the Rust Standard Library.
Passes LLVM codegen and Emscripten link-time flags for exception
handling if and only if the panic strategy is `unwind`. Sets the
default panic strategy for Emscripten targets to `unwind`. Re-enables
tests that depend on unwinding support for Emscripten, including
`should_panic` tests.
Split the rustc target libraries into separate rustc-dev component
This is re-applies a squashed version of #64823 as well as including #65337 to fix bugs noted after merging the first PR.
The second PR is confirmed as fixing windows-gnu, and presumably also fixes other platforms, such as musl (i.e. #65335 should be fixed); `RUSTUP_DIST_SERVER=https://dev-static.rust-lang.org rustup toolchain install nightly-2019-10-16` can be installed to confirm that this is indeed the case.
Remove `src/llvm-emscripten` submodule
With #65251 landed there's no need to build two LLVM backends and ship
them with rustc, every target we have now uses the same LLVM backend!
This removes the `src/llvm-emscripten` submodule and additionally
removes all support from rustbuild for building the emscripten LLVM
backend. Multiple codegen backend support is left in place for now, and
this is intended to be an easy 10-15 minute win on CI times by avoiding
having to build LLVM twice.
With #65251 landed there's no need to build two LLVM backends and ship
them with rustc, every target we have now uses the same LLVM backend!
This removes the `src/llvm-emscripten` submodule and additionally
removes all support from rustbuild for building the emscripten LLVM
backend. Multiple codegen backend support is left in place for now, and
this is intended to be an easy 10-15 minute win on CI times by avoiding
having to build LLVM twice.
