Add sanitizer support for modern iOS platforms
asan and tsan generally support iOS, but that previously wasn't configured in rust. This only adds support for the simulator architectures, and arm64 device architecture, not the older 32 bit architectures.
asan and tsan generally support iOS, but that previously wasn't
configured in rust. This only adds support for the simulator
architectures, and arm64 device architecture, not the older 32 bit
architectures.
Include sanitizers supported by LLVM on s390x (asan, lsan, msan, tsan)
in the target definition, as well as in the compiletest supported list.
Build sanitizer runtime for the target. Enable sanitizers in the CI.
Handle non-existent upstream master branches in `x fmt`
People who do have a remote for `rust-lang/rust` but don't have the master branch checked out there used to get this error when running `x fmt`:
> fatal: ambiguous argument 'rust/master': unknown revision or path not in the working tree.
> Use '--' to separate paths from revisions, like this:
> 'git <command> [<revision>...] -- [<file>...]'
> rust/master
Which is not exactly helpful.
Now, we fall back to `origin/master` (hoping that at least that remote exists) for that case. If there is still some error, we just fall back to `x fmt .` and print a warning.
r? `@jyn514`
Historically, Rust's Fuchsia targets have been labeled x86_64-fuchsia
and aarch64-fuchsia. However, they should technically contain vendor
information. This CL changes Fuchsia's target triples to include the
"unknown" vendor since Clang now does normalization and handles all
triple spellings.
This was previously attempted in #90510, which was closed due to
inactivity.
Make LLD build forward-compatible with LLVM 16
Switch to using the cmake module instead of llvm-config. I believe this also removes the need for llvm-config-wrapper.
LLVM_CONFIG_PATH is no longer supported as of LLVM 16, switch to
using the cmake module instead.
We separately return the llvm-config and cmake directory paths,
because llvm-config always refers to the host binary, while
the cmake directory is for the target triple.
LLVM does this itself since 606cb8548a1b7763e0c8489c5efe66803a7ede72,
and 14 is no longer the correct standard when building lld 16,
causing build failures.
On stable, our artifacts are uploaded with the raw version number (e.g.,
1.65.0), not the channel. This adjusts our detection logic to use the
version number from src/version when we detect the stable channel.
Use BOLT in CI to optimize LLVM
This PR adds an optimization step in the Linux `dist` CI pipeline that uses [BOLT](https://github.com/llvm/llvm-project/tree/main/bolt) to optimize the `libLLVM.so` library built by boostrap.
Steps:
- [x] Use LLVM 15 as a bootstrap compiler and use it to build BOLT
- [x] Compile LLVM with support for relocations (`-DCMAKE_SHARED_LINKER_FLAGS="-Wl,-q"`)
- [x] Gather profile data using instrumented LLVM
- [x] Apply profile to LLVM that has already been PGOfied
- [x] Run with BOLT profiling on more benchmarks
- [x] Decide on the order of optimization (PGO -> BOLT?)
- [x] Decide how we should get `bolt` (currently we use the host `bolt`)
- [x] Clean up
The latest perf results can be found [here](https://github.com/rust-lang/rust/pull/94381#issuecomment-1258269440). The current CI build time with BOLT applied is around 1h 55 minutes.
fix: use git-commit-info for version information
Fixes#33286.
Fixes#86587.
This PR changes the current `git-commit-hash` file that `./x.py` dist puts in the `rustc-{version}-src.tar.{x,g}z` to contain the hash, the short hash, and the commit date from which the tarball was created, assuming git was available when it was. It uses this for reading the version so that rustc has all the appropriate metadata.
# Testing
Testing this is kind of a pain. I did it with something like
```sh
./x.py dist # ensure that `ignore-git` is `false` in config.toml
cp ./build/dist/rustc-1.65.0-dev-src.tar.gz ../rustc-1.65.0-dev-src.tar.gz
cd .. && tar -xzf rustc-1.65.0-dev-src && cd rustc-1.65.0-dev-src
./x.py build
```
Then, the output of `rustc -vV` with the stage1 compiler should have the `commit-hash` and `commit-date` fields filled, rather than be `unknown`. To be completely sure, you can use `rustc --sysroot` with the stdlib that the original `./x.py dist` made, which will require that the metadata matches.
Fix a bunch of typo
This PR will fix some typos detected by [typos].
I only picked the ones I was sure were spelling errors to fix, mostly in
the comments.
[typos]: https://github.com/crate-ci/typos
This PR will fix some typos detected by [typos].
I only picked the ones I was sure were spelling errors to fix, mostly in
the comments.
[typos]: https://github.com/crate-ci/typos
See comment added for details on the test builder restriction. This is primarily
intended for macOS CI, but is likely to be a slight win on other builders too.
This verifies if the HEAD sha matches with the detected LLVM SHA, and if not,
permits usage of the detected LLVM. Otherwise, we fallback on regular
non-downloaded LLVM (currently still cached with sccache, though that's still
10+ minutes on macOS).
When building LLVM/LLD as part of a build that asks LLVM to generate profiles, e.g. when
doing PGO, cmake or clang-cl don't automatically link clang's profiler runtime in,
causing undefined reference errors at link-time.
We do that manually, by adding clang's resource library folder to the library search path:
- for LLVM itself, by extending the linker args that `rustc_llvm`'s build script
uses, to avoid the linker errors when linking `rustc_driver`.
- for LLD, by extending cmake's linker flags during the LLD build step.
Allow using `download-ci-llvm = true` outside the git checkout
`@bjorn3` noticed that this is already allowed today when download-llvm is disabled, but breaks with it enabled:
```
$ ./rust2/x.py build
fatal: not a git repository (or any of the parent directories): .git
thread 'main' panicked at 'command did not execute successfully: "git" "rev-list" "--author=bors@rust-lang.org" "-n1" "--first-parent" "HEAD" "--" "/home/jnelson/rust-lang/rust2/src/llvm-project" "/home/jnelson/rust-lang/rust2/src/bootstrap/download-ci-llvm-stamp" "/home/jnelson/rust-lang/rust2/src/version"
expected success, got: exit status: 128', src/bootstrap/native.rs:134:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Support it too for consistency. It's unclear to me when anyone would need to use this, but `@bjorn3`
feels we should support it, and it's not much additional effort to get it working.
LLVM tools should also be patched, since they are used in some tests,
specially,
- src/test/run-make-fulldeps/cross-lang-lto (llvm-ar)
- src/test/run-make-fulldeps/cross-lang-lto-upstream-rlibs (llvm-ar)
- src/test/run-make-fulldeps/issue-64153 (llvm-objdump)
To be more future proof, we should patch all binaries in `bin`.
@bjorn3 noticed that this is already allowed today when download-llvm is disabled, but breaks with it enabled:
```
$ ./rust2/x.py build
fatal: not a git repository (or any of the parent directories): .git
thread 'main' panicked at 'command did not execute successfully: "git" "rev-list" "--author=bors@rust-lang.org" "-n1" "--first-parent" "HEAD" "--" "/home/jnelson/rust-lang/rust2/src/llvm-project" "/home/jnelson/rust-lang/rust2/src/bootstrap/download-ci-llvm-stamp" "/home/jnelson/rust-lang/rust2/src/version"
expected success, got: exit status: 128', src/bootstrap/native.rs:134:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Support it too for consistency. It's unclear to me when anyone would need to use this, but @bjorn3
feels we should support it, and it's not much additional effort to get it working.
This also updates a bunch of other git commands that were similarly depending on the current directory.
Fix several issues during cross compiling
- When cross compiling LLVM on an arm64 macOS machine to x86_64, CMake will produce universal binaries by default, causing link errors. Explicitly set `CMAKE_OSX_ARCHITECTURES` to the one single target architecture so that the executables and libraries will be single architecture.
- When cross compiling rustc with `llvm.clang = true`, `CLANG_TABLEGEN` has to be set to the host `clang-tblgen` executable to build clang.
When cross compiling on macOS with `llvm.link-shared` enabled,
the symlink creation will fail after compiling LLVM for the target
architecture, because it will attempt to create the symlink in the
host LLVM directory, which was already created when being built.
This commit changes the symlink path to the actual LLVM output.
When cross compiling LLVM on an arm64 machine to x86_64, CMake will
produce universal binaries by default, causing link errors. Explicitly
set CMAKE_OSX_ARCHITECTURES to the one single target architecture.
Allow macOS to build LLVM as shared library
Inspired by how [homebrew](https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/llvm.rb) builds and distributes llvm, here we manually create a symlink with a versioned dylib path to make `llvm-config` work properly. Note, the resulting `rustc` executable and `librustc_driver-<hash>.dylib` still links to the un-versioned `libLLVM.dylib` as expected when distributed in the final output. I have confirmed this by checking `otool -L` on both binaries.
After the change, enabling `llvm.link-shared` and `llvm.thin-lto` will be possible on macOS.
