adding autodiff tests
I'd like to get started with upstreaming some tests, even though I'm still waiting for an answer on how to best integrate the enzyme pass. Can we therefore temporarily support the -Z llvm-plugins here without too much effort? And in that case, how would that work? I saw you can do remapping, e.g. `rust-src-base`, but I don't think that will give me the path to libEnzyme.so. Do you have another suggestion?
Other than that this test simply checks that the derivative of `x*x` is `2.0 * x`, which in this case is computed as
`%0 = fadd fast double %x.0.val, %x.0.val`
(I'll add a few more tests and move it to an autodiff folder if we can use the -Z flag)
r? ``@jieyouxu``
Locally at least `-Zllvm-plugins=${PWD}/build/x86_64-unknown-linux-gnu/enzyme/build/Enzyme/libEnzyme-19.so` seems to work if I copy the command I get from x.py test and run it manually. However, running x.py test itself fails.
Tracking:
- https://github.com/rust-lang/rust/issues/124509
Zulip discussion: https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Enzyme.20build.20changes
[`compiletest`-related cleanups 2/7] Feed stage number to compiletest directly
Reference for overall changes: https://github.com/rust-lang/rust/pull/136437
Part **2** of **7** of the *`compiletest`-related cleanups* PR series.
### Summary
- Pass stage number via new `--stage` compiletest flag directly from bootstrap, instead of deriving that info in compiletest by doing gymnastics on `--stage-id`.
- Just a cleanup, should have no functional changes.
r? bootstrap
Add amdgpu target
Add amdgpu target to rustc and enable the LLVM target.
Fix compiling `core` with the amdgpu:
The amdgpu backend makes heavy use of different address spaces. This
leads to situations, where a pointer in one addrspace needs to be casted
to a pointer in a different addrspace. `bitcast` is invalid for this
case, `addrspacecast` needs to be used.
Fix compilation failures that created bitcasts for such cases by
creating pointer casts (which creates an `addrspacecast` under the hood)
instead.
MCP: https://github.com/rust-lang/compiler-team/issues/823
Tracking issue: #135024
Kinda related to the original amdgpu tracking issue #51575 (though that one has been closed for a while).
Update bootstrap compiler and rustfmt
The rustfmt version we previously used formats things differently from what the latest nightly rustfmt does. This causes issues for subtrees that get formatted both in-tree and in their own repo. Updating the rustfmt used in-tree solves those issues. Also bumped the bootstrap compiler as the stage0 update command always updates both at the same
time.
Rollup of 5 pull requests
Successful merges:
- #134679 (Windows: remove readonly files)
- #136213 (Allow Rust to use a number of libc filesystem calls)
- #136530 (Implement `x perf` directly in bootstrap)
- #136601 (Detect (non-raw) borrows of null ZST pointers in CheckNull)
- #136659 (Pick the max DWARF version when LTO'ing modules with different versions )
r? `@ghost`
`@rustbot` modify labels: rollup
Rollup of 6 pull requests
Successful merges:
- #136640 (Debuginfo for function ZSTs should have alignment of 8 bits, not 1 bit)
- #136648 (Add a missing `//@ needs-symlink` to `tests/run-make/libs-through-symlinks`)
- #136651 (Label mismatched parameters at the def site for foreign functions)
- #136691 (Remove Linkage::Private and Linkage::Appending)
- #136692 (add module level doc for bootstrap:utils:exec)
- #136700 (i686-unknown-hurd-gnu: bump baseline CPU to Pentium 4)
r? `@ghost`
`@rustbot` modify labels: rollup
Enable "jump to def" feature on rustc docs
This PR enables the rustdoc "jump to def" feature which is visible on the source code pages.
r? ``@oli-obk``
bootstrap: add wrapper macros for `feature = "tracing"`-gated `tracing` macros
Follow-up to https://github.com/rust-lang/rust/pull/136091#discussion_r1930219425.
- Add wrapper macros for `error!`, `warn!`, `info!`, `debug!` and `trace!`, which `cfg(feature = "tracing")`-gates the underlying `tracing` macros. They expand to nothing if `"tracing"` feature is not enabled.
- This is not done for `span!` or `event!` because they can return span guards, and you can't really wrap that.
- This is also not possible for `tracing::instrument` attribute proc-macro unless you use another attribute proc-macro to wrap that.
It's not *great*, because `tracing::instrument` and `tracing::{span,event}` can't be wrapped this way.
Can test locally with:
```bash
$ BOOTSTRAP_TRACING=bootstrap=TRACE ./x check src/bootstrap/
```
r? ``@onur-ozkan`` (or reroll)
- Add wrapper macros for `error!`, `warn!`, `info!`, `debug!` and
`trace!`, which `cfg(feature = "tracing")`-gates the underlying
`tracing` macros.
- This is not done for `span!` or `event!` because they can return span
guards, and you can't really wrap that.
- This is also not possible for `tracing::instrument` attribute
proc-macro unless you use another attribute proc-macro to wrap that.
bootstrap: only build `crt{begin,end}.o` when compiling to MUSL
only MUSL needs those objects and trying to compile them to other targets, e.g. Windows or macOS, will produce C compilation errors
check the target before shelling out to the C compiler and tweak `make_run` to skip the actual C compilation when the target is not MUSL
fixes#135782
see the linked issue for additional context
only MUSL needs those objects and trying to compile them to other
targets, e.g. Windows or macOS, will produce C compilation errors
check the target before shelling out to the C compiler and tweak
`make_run` to skip the actual C compilation when the target is not MUSL
fixes#135782
Autodiff Upstreaming - rustc_codegen_ssa, rustc_middle
This PR should not be merged until the rustc_codegen_llvm part is merged.
I will also alter it a little based on what get's shaved off from the cg_llvm PR,
and address some of the feedback I received in the other PR (including cleanups).
I am putting it already up to
1) Discuss with `@jieyouxu` if there is more work needed to add tests to this and
2) Pray that there is someone reviewing who can tell me why some of my autodiff invocations get lost.
Re 1: My test require fat-lto. I also modify the compilation pipeline. So if there are any other llvm-ir tests in the same compilation unit then I will likely break them. Luckily there are two groups who currently have the same fat-lto requirement for their GPU code which I have for my autodiff code and both groups have some plans to enable support for thin-lto. Once either that work pans out, I'll copy it over for this feature. I will also work on not changing the optimization pipeline for functions not differentiated, but that will require some thoughts and engineering, so I think it would be good to be able to run the autodiff tests isolated from the rest for now. Can you guide me here please?
For context, here are some of my tests in the samples folder: https://github.com/EnzymeAD/rustbook
Re 2: This is a pretty serious issue, since it effectively prevents publishing libraries making use of autodiff: https://github.com/EnzymeAD/rust/issues/173. For some reason my dummy code persists till the end, so the code which calls autodiff, deletes the dummy, and inserts the code to compute the derivative never gets executed. To me it looks like the rustc_autodiff attribute just get's dropped, but I don't know WHY? Any help would be super appreciated, as rustc queries look a bit voodoo to me.
Tracking:
- https://github.com/rust-lang/rust/issues/124509
r? `@jieyouxu`
Commands that end up invoking cc-rs, i.e. Cargo (through build scripts)
and cmake-rs don't need the CFLAGS from cc-rs itself, as they will just
end up as duplicates.
We do still choose to pass them in certain places, but now it's at least
clear which flags are default, and which flags are extra flags added on.
