I spent a while debugging a strage linker error about an outdated `glibc` version, only to discover that it was caused by a stale `obj` directory. It wasn't obviously to be that using the same obj dir with multiple Docker images (for the same target triple) could be a problem.
This commit adds a note to the README, which should hopefully be helpful to anyone else who runs into this issue.
This commit updates the `wasi-libc` repository used to build the
wasm32-wasi target's libstd to ensure that both libstd and libc are
using the same wasi snapshot version.
LLVM 7 is over a year old, which should be plenty for compatibility. The
last LLVM 6 holdout was llvm-emscripten, which went away in #65501.
I've also included a fix for LLVM 8 lacking `MemorySanitizerOptions`,
which was broken by #66522.
Misc CI improvements
This PR contains some misc improvements to our CI configuration:
* The environment variables for MinGW builders were greatly simplified, with just `CUSTOM_MINGW=1` to tell the install scripts to install the vendored copy. All the others (`MINGW_URL`, `MINGW_DIR`, `MINGW_ARCHIVE` and `MSYS_BITS`) are detected either from the builder name or the environment.
* Collecting CPU stats and running the build were moved into scripts.
* Toolstate scripts validation was previously a separate step, ran just when `IMAGE=mingw-check`. This moves the validation code inside the actual image.
* Vendored copies are now fetched from https://ci-mirrors.rust-lang.org instead of directly from the bucket.
r? @alexcrichton
CentOS 5 only supports SSLv3 without SNI, and to get newer protocols
working we need to download and compile OpenSSL and cURL from our
mirror. Because of that, we can't use the CDN, as CloudFront requires
TLSv1 with SNI.
This commit changes the dist-x86_64-linux image to bypass the CDN for
OpenSSL and cURL.
This updates the libc that the `wasm32-wasi` target links against to the
latest revision, mostly just bringing in minor bug fixes and minor wasm
size improvements.
Upload toolstates.json to rust-lang-ci2
This PR does two things:
* Following up with https://github.com/rust-lang/rust/pull/65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended.
* Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot.
r? @alexcrichton
cc @RalfJung
Before this commit toolstates.json was stored in /tmp and it wasn't
mounted outside the build container. That caused uploading the file in
the upload-artifacts task to fail, as the file was missing on the host.
Mounting /tmp/toolstates.json alone is not the best approach: if the
file is missing when the container is started the Docker engine will
create a *directory* named /tmp/toolstates.json.
The Docker issue could be solved by pre-creating an empty file named
/tmp/toolstates.json, but doing that could cause problems if bootstrap
fails to generate the file and the toolstate scripts receive an empty
JSON.
The approach I took in this commit is to instead mount a /tmp/toolstate
directory inside Docker, and create the toolstates.json file in it. That
also required a small bootstrap change to ensure the directory is
created if it's missing.
With #65251 landed there's no need to build two LLVM backends and ship
them with rustc, every target we have now uses the same LLVM backend!
This removes the `src/llvm-emscripten` submodule and additionally
removes all support from rustbuild for building the emscripten LLVM
backend. Multiple codegen backend support is left in place for now, and
this is intended to be an easy 10-15 minute win on CI times by avoiding
having to build LLVM twice.
Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux
#62896 was caused by the usage of the GCC 5.2.0 toolchain, which was released back in 2015 and may have bugs affecting LLVM 9.
This PR upgrade GCC to 8.3.0 from 5.2.0, glibc from 1.16.0 to 1.17.0 and crosstool-ng to 1.24.0 only for dist-armv7-linux.
Fixes#62896
r? @alexcrichton
- Compatible with Emscripten 1.38.46-upstream or later upstream.
- Refactors the Emscripten target spec to share code with other wasm
targets.
- Replaces the old incorrect wasm32 C call ABI with the correct one,
preserving the old one as wasm32_bindgen_compat for wasm-bindgen
compatibility.
- Updates the varargs ABI used by Emscripten and deletes the old one.
- Removes the obsolete wasm32-experimental-emscripten target.
- Uses EMCC_CFLAGS on CI to avoid the timeout problems with #63649.
- Refactors the Emscripten target spec to share code with other wasm
targets.
- Replaces the incorrect wasm32 C call ABI with the old asmjs
version, which is correct for both wasm32 and JS.
- Updates the varargs ABI used by Emscripten and deletes the old one.
- Removes the obsolete wasm32-experimental-emscripten target.
- Temporarily makes Emscripten targets use panic=abort by default
because supporting unwinding will require an LLVM patch.
This commit disables LLVM/debug assertions in our 5 slowest builders:
* i686-gnu
* i686-gnu-nopt
* i686-msvc-1
* i686-msvc-2
* x86_64-msvc-cargo
This is reducing the amount of test coverage for LLVM/debug assertions,
but we're just unfortunately running out of time on CI too many times.
Some test builds have shown that i686-gnu drops nearly an hour of CI
time by disabling these two assertions. Perhaps when we eventually get
4-core machines we can reenable these, but for now turn them off and
hook them up to the tracking issue at #59637 which will ideally be
repurposes to tracking all of these.
Currently mirrors are stored in the rust-lang-ci2 S3 bucket along with
CI toolchains. This is problematic for multiple reasons:
- CI IAM credentials are allowed to both edit and delete those files.
A malicious user gaining access to those credentials would be able to
change our mirrored dependencies, possibly backdooring the compiler.
- Contents of the rust-lang-ci2 bucket are disposable except for the
mirrors' content. When we implement backups for S3 buckets we'd have
to replicate just that part of the bucket, complicating the backup
logic and increasing the chance of mistakes. A standalone bucket will
be way easier to backup.
This commit switches our CI to use the new rust-lang-ci-mirrors bucket.
Added support for armv7-unknown-linux-gnueabi/musleabi
Fixes#63101
Some things that are not done and I hope someone can help me with:
* During the ci build of `armv7-unknown-linux-gnueabi` `openssl` must be built (to build cargo) but `openssl` does not yet support this target. This feels slightly like a chicken-and-egg problem, any feedback is welcome.
* Should I add any tests for any of these targets?
