Also removes stage1, stage2 cfgs being passed to rustc to ensure that
stage1 and stage2 are only differentiated as a group (i.e., only through
not bootstrap).
use SecRandomCopyBytes on macOS in Miri
This is a hack to fix https://github.com/rust-lang/miri/issues/686: on macOS, rustc will open `/dev/urandom` to initialize a `HashMap`. That's quite hard to emulate properly in Miri without a full-blown implementation of file descriptors. However, Miri needs an implementation of `SecRandomCopyBytes` anyway to support [getrandom](https://crates.io/crates/getrandom), so using it here should work just as well.
This will only have an effect when libstd is compiled specifically for Miri, but that will generally be the case when people use `cargo miri`.
This is clearly a hack, so I am opening this to start a discussion about whether we are okay with such a hack or not.
Cc @oli-obk
This is result of squashing two revert commits:
Revert "compile all crates under test w/ -Zemit-stack-sizes"
This reverts commit 7d365cf27f.
Revert "bootstrap: build compiler-builtins with -Z emit-stack-sizes"
This reverts commit 8b8488ce8f.
This commit adds a new wasm32-based target distributed through rustup,
supported in the standard library, and implemented in the compiler. The
`wasm32-unknown-wasi` target is intended to be a WebAssembly target
which matches the [WASI proposal recently announced.][LINK]. In summary
the WASI target is an effort to define a standard set of syscalls for
WebAssembly modules, allowing WebAssembly modules to not only be
portable across architectures but also be portable across environments
implementing this standard set of system calls.
The wasi target in libstd is still somewhat bare bones. This PR does not
fill out the filesystem, networking, threads, etc. Instead it only
provides the most basic of integration with the wasi syscalls, enabling
features like:
* `Instant::now` and `SystemTime::now` work
* `env::args` is hooked up
* `env::vars` will look up environment variables
* `println!` will print to standard out
* `process::{exit, abort}` should be hooked up appropriately
None of these APIs can work natively on the `wasm32-unknown-unknown`
target, but with the assumption of the WASI set of syscalls we're able
to provide implementations of these syscalls that engines can implement.
Currently the primary engine implementing wasi is [wasmtime], but more
will surely emerge!
In terms of future development of libstd, I think this is something
we'll probably want to discuss. The purpose of the WASI target is to
provide a standardized set of syscalls, but it's *also* to provide a
standard C sysroot for compiling C/C++ programs. This means it's
intended that functions like `read` and `write` are implemented for this
target with a relatively standard definition and implementation. It's
unclear, therefore, how we want to expose file descriptors and how we'll
want to implement system primitives. For example should `std::fs::File`
have a libc-based file descriptor underneath it? The raw wasi file
descriptor? We'll see! Currently these details are all intentionally
hidden and things we can change over time.
A `WasiFd` sample struct was added to the standard library as part of
this commit, but it's not currently used. It shows how all the wasi
syscalls could be ergonomically bound in Rust, and they offer a possible
implementation of primitives like `std::fs::File` if we bind wasi file
descriptors exactly.
Apart from the standard library, there's also the matter of how this
target is integrated with respect to its C standard library. The
reference sysroot, for example, provides managment of standard unix file
descriptors and also standard APIs like `open` (as opposed to the
relative `openat` inspiration for the wasi ssycalls). Currently the
standard library relies on the C sysroot symbols for operations such as
environment management, process exit, and `read`/`write` of stdio fds.
We want these operations in Rust to be interoperable with C if they're
used in the same process. Put another way, if Rust and C are linked into
the same WebAssembly binary they should work together, but that requires
that the same C standard library is used.
We also, however, want the `wasm32-unknown-wasi` target to be
usable-by-default with the Rust compiler without requiring a separate
toolchain to get downloaded and configured. With that in mind, there's
two modes of operation for the `wasm32-unknown-wasi` target:
1. By default the C standard library is statically provided inside of
`liblibc.rlib` distributed as part of the sysroot. This means that
you can `rustc foo.wasm --target wasm32-unknown-unknown` and you're
good to go, a fully workable wasi binary pops out. This is
incompatible with linking in C code, however, which may be compiled
against a different sysroot than the Rust code was previously
compiled against. In this mode the default of `rust-lld` is used to
link binaries.
2. For linking with C code, the `-C target-feature=-crt-static` flag
needs to be passed. This takes inspiration from the musl target for
this flag, but the idea is that you're no longer using the provided
static C runtime, but rather one will be provided externally. This
flag is intended to also get coupled with an external `clang`
compiler configured with its own sysroot. Therefore you'll typically
use this flag with `-C linker=/path/to/clang-script-wrapper`. Using
this mode the Rust code will continue to reference standard C
symbols, but the definition will be pulled in by the linker configured.
Alright so that's all the current state of this PR. I suspect we'll
definitely want to discuss this before landing of course! This PR is
coupled with libc changes as well which I'll be posting shortly.
[LINK]:
[wasmtime]:
bootstrap: build crates under libtest with -Z emit-stack-sizes
Please see the comment in the diff for the rationale.
This change adds a `.stack_sizes` linker section to `libcompiler_builtins.rlib`
but this section is discarded by the linker by default so it won't affect the
binary size of most programs. It will, however, negatively affect the binary
size of programs that link to a recent release of the `cortex-m-rt` crate
because of the linker script that crate provides, but I have proposed a PR
(rust-embedded/cortex-m-rt#186) to solve the problem (which I originally
introduced :-)).
This change does increase the size of the `libcompiler_builtins.rlib` artifact we
distribute but the increase is in the order of (a few) KBs.
r? @alexcrichton
Rustdoc remove old style files
Reopening of #56577 (which I can't seem to reopen...).
I made the flag unstable so with this change, what was blocking the PR is now gone I assume.
Use lld directly for Fuchsia target
Fuchsia already uses lld as the default linker, so there's no reason
to always invoke it through Clang, instead we can simply invoke lld
directly and pass the set of flags that matches Clang.
Fuchsia already uses lld as the default linker, so there's no reason
to always invoke it through Clang, instead we can simply invoke lld
directly and pass the set of flags that matches Clang.
* Make it influence the behavior of the compiled rustc, rather than
just the rustc build system. That is, if verify_llvm_ir=true,
even manual invocations of the built rustc will verify LLVM IR.
* Enable verification of LLVM IR in CI, for non-deploy and
deploy-alt builds. This is similar to how LLVM assertions are
handled.
This commit updates the debuginfo that is encoded in all of our released
artifacts by default. Currently it has paths like `/checkout/src/...` but these
are a little inconsistent and have changed over time. This commit instead
attempts to actually define the file paths in our debuginfo to be consistent
between releases.
All debuginfo paths are now intended to be `/rustc/$sha` where `$sha` is the git
sha of the released compiler. Sub-paths are all paths into the git repo at that
`$sha`.
Use the correct crt*.o files when linking musl targets.
This is supposed to support optionally using the system copy of musl
libc instead of the included one if supported. This currently only
affects the start files, which is enough to allow building rustc on musl
targets.
Most of the changes are analogous to crt-static.
Excluding the start files is something musl based distributions usually patch into their copy of rustc:
- eb064c8/community/rust/musl-fix-linux_musl_base.patch
- 77400fc/srcpkgs/rust/patches/link-musl-dynamically.patch
For third-party distributions that not yet carry those patches it would be nice if it was supported without the need to patch upstream sources.
## Reasons
### What breaks?
Some start files were missed when originally writing the logic to swap in musl start files (gcc comes with its own start files, which are suppressed by -nostdlib, but not manually included later on). This caused #36710, which also affects rustc with the internal llvm copy or any other system libraries that need crtbegin/crtend.
### How is it fixed?
The system linker already has all the logic to decide which start files to include, so we can just defer to it (except of course if it doesn't target musl).
### Why is it optional?
In #40113 it was first tried to remove the start files, which broke compiling musl-targeting static binaries with a glibc-targeting compiler. This is why it eventually landed without removing the start files. Being an option side-steps the issue.
### Why are the start files still installed?
This has the nice side-effect, that the produced rust-std-* binaries can still be used by on a glibc-targeting system with a rustc built against glibc.
## Does it work?
With the following build script (using [musl-cross-make](https://github.com/richfelker/musl-cross-make)): https://shadowice.org/~mixi/rust-musl/build.sh, I was able to cross-compile a musl-host musl-targeting rustc on a glibc-based system. The resulting binaries are at https://shadowice.org/~mixi/rust-musl/binaries/. This also requires #50103 and #50104 (which are also applied to the branch the build script uses).
Currently on CI we predominately compile LLVM with the default system compiler
which means gcc on Linux, some version of Clang on OSX, MSVC on Windows, and
gcc on MinGW. This commit switches Linux, OSX, and Windows to all use Clang
6.0.0 to build LLVM (aka the C/C++ compiler as part of the bootstrap). This
looks to generate faster code according to #49879 which translates to a faster
rustc (as LLVM internally is faster)
The major changes here were to the containers that build Linux releases,
basically adding a new step that uses the previous gcc 4.8 compiler to compile
the next Clang 6.0.0 compiler. Otherwise the OSX and Windows scripts have been
updated to download precompiled versions of Clang 6 and configure the build to
use them.
Note that `cc` was updated here to fix using `clang-cl` with `cc-rs` on MSVC, as
well as an update to `sccache` on Windows which was needed to correctly work
with `clang-cl`. Finally the MinGW compiler is entirely left out here
intentionally as it's currently thought that Clang can't generate C++ code for
MinGW and we need to use gcc, but this should be verified eventually.
This commit updates CI configuration to inform rustbuild that it should print
out how long each step takes on CI. This'll hopefully allow us to track the
duration of steps over time and follow regressions a bit more closesly (as well
as have closer analysis of differences between two builds).
cc #48829
rustbuild: Remove ThinLTO-related configuration
This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.
This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
