Allow configuring jemalloc per target
In Ferrocene we're trying to switch from `./configure` to a predefined `config.toml` file. One of the limitations of doing that is the `rust.jemalloc` configuration option, which we need to conditionally disable based on the target. This PR adds a `target.$tuple.jemalloc` option to override `rust.jemalloc` to make that possible.
boostrap: skip no_std targets in Std doc step
This fixes a bug that currently prevents us from adding no_std library targets to rustc in nixpkgs (https://github.com/NixOS/nixpkgs/pull/382166).
When running `./x.py doc`, the `Std` doc step generally fails for no_std targets, logs: https://gist.github.com/niklaskorz/fb83f9503ce19b75e8b1af02cdebd592
Skipping no_std targets in this step will allow using no_std targets such as `bpfel-unknown-none` together with other targets in the same config without blocking the doc generator for them, e.g.
```
./configure --release-channel=stable --tools=rustc,rustdoc,rust-analyzer-proc-macro-srv --build=aarch64-apple-darwin --host=aarch64-apple-darwin --target=aarch64-apple-darwin,bpfel-unknown-none
./x.py doc
```
Logs with this fix applied: https://gist.github.com/niklaskorz/cdd50aaea33ede579f737434286d800b
Pass vendored sources from bootstrap to generate-copyright
In addition to doing the vendoring in bootstrap, this PR also loads the list of manifests to parse from bootstrap (instead of hardcoding a smaller list in generate-copyright). This is best reviewed commit-by-commit.
Fixes https://github.com/rust-lang/rust/issues/136955
Add profiling of bootstrap commands using Chrome events
Since we now have support for tracing in bootstrap, and the execution of most commands is centralized within a few functions, it's quite trivial to also trace command execution, and visualize it using the Chrome profiler. This can be helpful both to profile what takes time in bootstrap and also to get a visual idea of what happens in a given bootstrap invocation (since the execution of external commands is usually the most interesting thing).
This is how it looks:
I first tried to use [tracing-flame](https://github.com/tokio-rs/tracing/tree/master/tracing-flame), but the output wasn't very useful, because the event/stackframe names were bootstrap code locations, instead of the command contents.
r? ``@jieyouxu``
Compiletest should not inherit all host RUSTFLAGS
I told ``@rhelmot`` to do this in #134913. But it's not correct; compiletest shouldn't inherit RUSTFLAGS at all.
Pass a single new --host-rustcflags to compiletest instead, without overwriting any existing arguments.
Fixes the following failure, which only happens when building llvm from source and then running `x test --stage 1 ui-fulldeps`:
```
diff --git a/tests/ui-fulldeps/fluent-messages/test.stderr b/tests/ui-fulldeps/fluent-messages/test.stderr
index 0b3bb14ce51..978ac46c5a2 100644
--- a/tests/ui-fulldeps/fluent-messages/test.stderr
+++ b/tests/ui-fulldeps/fluent-messages/test.stderr
``@@`` -1,3 +1,8 ``@@``
+warning[E0602]: unknown lint: `linker_messages`
+ |
+ = note: requested on the command line with `-A linker_messages`
+ = note: `#[warn(unknown_lints)]` on by default
```
See https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/.E2.9C.94.20unknown.20lint.3A.20.60linker_messages.60.20when.20blessing.20tests.20on.20.2E.2E.2E for more context.
"stage 1" for fulldeps means "compile with stage 0, link against stage 1".
But this code wanted to switch on the compiler that's building, not the
compiler that's being tested. Fix the check.
Previously, it would fail with a warning about linker-messages:
```
--- stderr -------------------------------
warning[E0602]: unknown lint: `linker_messages`
|
= note: requested on the command line with `-A linker_messages`
= note: `#[warn(unknown_lints)]` on by default
```
I told rhelmot to do this in #134913. But it's not correct; compiletest
shouldn't inherit RUSTFLAGS at all.
Pass a single new --host-rustcflags to compiletest instead, without overwriting any
existing arguments.
Fixes the following failure, which only happens when building llvm from
source and then running `x test --stage 1 ui-fulldeps`:
```
diff --git a/tests/ui-fulldeps/fluent-messages/test.stderr b/tests/ui-fulldeps/fluent-messages/test.stderr
index 0b3bb14ce51..978ac46c5a2 100644
--- a/tests/ui-fulldeps/fluent-messages/test.stderr
+++ b/tests/ui-fulldeps/fluent-messages/test.stderr
@@ -1,3 +1,8 @@
+warning[E0602]: unknown lint: `linker_messages`
+ |
+ = note: requested on the command line with `-A linker_messages`
+ = note: `#[warn(unknown_lints)]` on by default
```
use cc archiver as default in `cc2ar`
We should remove entire `cc2ar` but `cc` doesn't seem to cover all the conditions that `cc2ar` handles. For now, I replaced the `else` logic only, which is a bit hacky and unstable.
Fixes#136759
(S)ccache can be useful for more things that just LLVM. For example, we will soon want to use it also for GCC, and theoretically also for building stage0 Rust tools.
improve host/cross target checking
Using an invalid equality operator on `builder.config.build !=/==` can be hard to detect in reviews (which is quite dangerous). Replaced them with `is_host_target`, which is much clearer as it explicitly states what it does.
We should remove entire `cc2ar` but `cc` doesn't seem to cover all the conditions that `cc2ar` handles.
For now, I replaced the `else` logic only, which is a bit hacky and unstable.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
adding autodiff tests
I'd like to get started with upstreaming some tests, even though I'm still waiting for an answer on how to best integrate the enzyme pass. Can we therefore temporarily support the -Z llvm-plugins here without too much effort? And in that case, how would that work? I saw you can do remapping, e.g. `rust-src-base`, but I don't think that will give me the path to libEnzyme.so. Do you have another suggestion?
Other than that this test simply checks that the derivative of `x*x` is `2.0 * x`, which in this case is computed as
`%0 = fadd fast double %x.0.val, %x.0.val`
(I'll add a few more tests and move it to an autodiff folder if we can use the -Z flag)
r? ``@jieyouxu``
Locally at least `-Zllvm-plugins=${PWD}/build/x86_64-unknown-linux-gnu/enzyme/build/Enzyme/libEnzyme-19.so` seems to work if I copy the command I get from x.py test and run it manually. However, running x.py test itself fails.
Tracking:
- https://github.com/rust-lang/rust/issues/124509
Zulip discussion: https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Enzyme.20build.20changes
