Metadata::modified works in all platforms supported by the filetime
crate. This changes brings rustbuild a tiny bit closer towards dropping
the filetime dependency.
adapt static-nobundle test to use llvm-nm
No functional changes intended.
This updates the test case to use llvm-nm instead of the system nm.
This fixes an instance over at the experimental build of rustc with HEAD LLVM:
https://buildkite.com/llvm-project/rust-llvm-integrate-prototype/builds/8380#ef6f41b5-8595-49a6-be37-0eff80e0ccb5
It is related to https://github.com/rust-lang/rust/pull/94001.
The issue is that this test uses the system nm, which may not be recent
enough to understand the update to uwtable. This replaces the test to
use the llvm-nm that should be recent enough (consistent with the LLVM
sources we use to build rustc).
bootstrap: tidy up flag handling for llvm build
This tidies up the logic in `src/bootstrap/native.rs` such that:
- `CMAKE_*_LINKER_FLAGS` is not overridden if we add to it twice.
- `CMAKE_*_FLAGS` also include the standard `*FLAGS` environment
variables, which CMake respects when we _don't_ set `CMAKE_*_FLAGS`.
- `llvm.ldflags` from `config.toml` appends to the ldflags Rust's
bootstrap logic adds, rather than replacing them.
It also takes a second stab at #89983 by moving `-static-libstdc++` to just be passed as a linker flag, since that's what it is.
Fixes#93880. Fixes#70468. Closes#89983.
Drop time dependency from bootstrap
This was only used for the inclusion of 'current' dates into our manpages, but
it is not clear that this is practically necessary. The manpage is essentially
never updated, and so we can likely afford to keep a manual date in these files.
It also seems possible to just omit it, but that may cause other tools trouble,
so avoid doing that for now.
This is largely done to reduce bootstrap complexity; the time crate is not particularly
small and in #92480 would have started pulling in num-threads, which does runtime
thread count detection. I would prefer to avoid that, so filing this to just drop the nearly
unused dependency entirely.
r? `@pietroalbini`
Rollup of 9 pull requests
Successful merges:
- #89926 (make `Instant::{duration_since, elapsed, sub}` saturating and remove workarounds)
- #90532 (More informative error message for E0015)
- #93810 (Improve chalk integration)
- #93851 (More practical examples for `Option::and_then` & `Result::and_then`)
- #93885 (bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download)
- #93886 (Stabilise inherent_ascii_escape (FCP in #77174))
- #93930 (add link to format_args! when mention it in docs)
- #93936 (Couple of driver cleanups)
- #93944 (Don't relabel to a team if there is already a team label)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download
I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.
To reproduce my error:
```
git checkout 8d7707f3c4
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404
failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46
```
This is my `config.toml`:
```
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2
[rust]
debug = true
```
To reproduce an error with this branch:
Change line 618 of bootstrap.py to
```
url = "rustc-builds-error404/{}".format(llvm_sha)
```
Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml.
```
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404
failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:
[llvm]
download-ci-llvm = false
Build completed unsuccessfully in 0:00:01
```
Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.
This tidies up the logic in `src/bootstrap/native.rs` such that:
- `CMAKE_*_LINKER_FLAGS` is not overridden if we add to it twice.
- `CMAKE_*_FLAGS` also include the standard `*FLAGS` environment
variables, which CMake respects when we _don't_ set `CMAKE_*_FLAGS`.
- `llvm.ldflags` from `config.toml` appends to the ldflags Rust's
bootstrap logic adds, rather than replacing them.
Fixes#93880.
Lazy type-alias-impl-trait
Previously opaque types were processed by
1. replacing all mentions of them with inference variables
2. memorizing these inference variables in a side-table
3. at the end of typeck, resolve the inference variables in the side table and use the resolved type as the hidden type of the opaque type
This worked okayish for `impl Trait` in return position, but required lots of roundabout type inference hacks and processing.
This PR instead stops this process of replacing opaque types with inference variables, and just keeps the opaque types around.
Whenever an opaque type `O` is compared with another type `T`, we make the comparison succeed and record `T` as the hidden type. If `O` is compared to `U` while there is a recorded hidden type for it, we grab the recorded type (`T`) and compare that against `U`. This makes implementing
* https://github.com/rust-lang/rfcs/pull/2515
much simpler (previous attempts on the inference based scheme were very prone to ICEs and general misbehaviour that was not explainable except by random implementation defined oddities).
r? `@nikomatsakis`
fixes#93411fixes#88236
This was only used for the inclusion of 'current' dates into our manpages, but
it is not clear that this is practically necessary. The manpage is essentially
never updated, and so we can likely afford to keep a manual date in these files.
It also seems possible to just omit it, but that may cause other tools trouble,
so avoid doing that for now.
bootstrap: prefer using '--config' over 'RUST_BOOTSTRAP_CONFIG'
Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
Closes: #93725
Rleated: #92260
Rerun bootstrap's build script when RUSTC changes
Previously, rustbuild would give strange errors if you tried to reuse the same build directory under two names:
```
$ mkdir tmp && cd tmp
$ ../x.py check
Building rustbuild
Finished dev [unoptimized] target(s) in 35.27s
Checking stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
^C
$ cd ..
$ mv tmp/build build
$ ./x.py check
Building rustbuild
Compiling bootstrap v0.0.0 (/home/jnelson/rust-lang/rust/src/bootstrap)
Finished dev [unoptimized] target(s) in 11.18s
failed to execute command: "/home/jnelson/rust-lang/rust/tmp/build/x86_64-unknown-linux-gnu/stage0/bin/rustc" "--target" "x86_64-unknown-linux-gnu" "--print" "target-libdir"
error: No such file or directory (os error 2)
```
This fixes the error. Reusing the same build directory is useful if you want to test path-related things in
bootstrap itself, without having to recompile it each time.
For good measure, this also reruns the build script when PATH changes.
Use shallow clones for submodules managed by rustbuild, not just bootstrap.py
I missed this in https://github.com/rust-lang/rust/pull/89757; it made
`x.py test src/bootstrap` very slow.
Make the pre-commit script pre-push instead
This should make it substantially less annoying, and hopefully more
people will find it useful. In particular, it will no longer run tidy
each time you run `git commit --amend` or rebase a branch.
This also warns if you have the old script in pre-commit; see the HACK
comment for details.
r? ````@Mark-Simulacrum```` cc ````@caass````
Previously, rustbuild would give strange errors if you tried to reuse the same build directory under two names:
```
$ mkdir tmp && cd tmp
$ ../x.py check
Building rustbuild
Finished dev [unoptimized] target(s) in 35.27s
Checking stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
^C
$ cd ..
$ mv tmp/build build
$ ./x.py check
Building rustbuild
Compiling bootstrap v0.0.0 (/home/jnelson/rust-lang/rust/src/bootstrap)
Finished dev [unoptimized] target(s) in 11.18s
failed to execute command: "/home/jnelson/rust-lang/rust/tmp/build/x86_64-unknown-linux-gnu/stage0/bin/rustc" "--target" "x86_64-unknown-linux-gnu" "--print" "target-libdir"
error: No such file or directory (os error 2)
```
This fixes the error. Reusing the same build directory is useful if you want to test path-things in
bootstrap itself, without having to recompile it each time.
For good measure, this also reruns the build script when PATH changes.
Hide failed command unless in verbose mode
This is particularly intended for invoking compiletest; the command line there
is long (3,350 characters on my system) and takes up a lot of screen real estate
for little benefit to the majority of those running bootstrap. This moves
printing it to verbose mode (-v must be passed) which means that it's still
possible to access when needed for debugging.
The main downside is that CI logs will by-default become less usable for
debugging (particularly) spurious failures, but it is pretty rare for us to
really need the information there -- it's usually fairly obvious what is being
run with a little investigation.
r? `@ehuss` as you've done some of the spurious failure investigations, so can
(hopefully) confirm my intuition that this won't seriously hinder them.
This is particularly intended for invoking compiletest; the command line there
is long (3,350 characters on my system) and takes up a lot of screen real estate
for little benefit to the majority of those running bootstrap. This moves
printing it to verbose mode (-v must be passed) which means that it's still
possible to access when needed for debugging.
The main downside is that CI logs will by-default become less usable for
debugging (particularly) spurious failures, but it is pretty rare for us to
really need the information there -- it's usually fairly obvious what is being
run with a little investigation.
build: dist: defer PlainSourceTarball
Apparently it changes some tool sources and invalidates their fingerprints, forcing us to build them several times (before and after vendoring sources).
I have not dug into why vendoring actually invalidates the figreprints, but moving the vendoring lower in the pipeline seems to avoid the issue.
I could imagine that we somehow write a .cargo/config somewhere which somehow makes subsequent builds use the vendored deps but I was not able to find anything.
I checked the sizes of generated archives pre and post patch and their are the same, so I hope there is no functional change.
Fixes#93033
x.py has support for excluding some steps from the invocation, but
unfortunately that's not granular enough: some steps have the same name
in different modules, and that prevents excluding only *some* of them.
As a practical example, let's say you need to run everything in `./x.py
test` except for the standard library tests, as those tests require IPv6
and need to be executed on a separate machine. Before this commit, if
you were to just run this:
./x.py test --exclude library/std
...the execution would fail, as that would not only exclude running the
tests for the standard library, it would also exclude generating its
documentation (breaking linkchecker).
This commit adds support for an optional module annotation in --exclude
paths, allowing the user to choose which module to exclude from:
./x.py test --exclude test::library/std
This maintains backward compatibility, but also allows for more ganular
exclusion. More examples on how this works:
| `--exclude` | Docs | Tests |
| ------------------- | ------- | ------- |
| `library/std` | Skipped | Skipped |
| `doc::library/std` | Skipped | Run |
| `test::library/std` | Run | Skipped |
Note that the new behavior only works in the `--exclude` flag, and not
in other x.py arguments or flags yet.
Building a dozen separate regexps for each test in compiletest consumes significant amounts of CPU cycles.
Using `RUSTC_FORCE_INCR_COMP_ARTIFACT_HEADER` stabilizes hashes calcuated for the individual tests so
no test-dependent normalization is needed. Hashes for the standard library still change so some
normalizations are still needed.
Add manifest docs fallback.
This adds a fallback so that the rustup manifest will contain the rust-docs component for all hosts. There is a mapping so that the docs that get downloaded are roughly close to the actual host. There inevitably will be things that don't match. Ideally the standard library docs would be the same for every platform (`cfg(doc)` goes a long way towards this), but there are still lots of minor differences.
Closes#69525
Apparently it changes some tool sources and invalidates their fingerprints, forcing us to build them several times (before and after vendoring sources).
I have not dug into why vendoring actually invalidates the figreprints, but the moving the vendoring lower in the pipeline seems to avoid the issue.
I could imagine that we somehow write a .cargo/config somewhere which somehow makes subsequent builds use the vendored deps but I was not able to find anything.
I checked the sizes of generated archives pre and post patch and their are the same, so I hope there is not functional change.
Fixes#93033
This builder is the slowest in the fleet. This should cut a considerable
amount of time. The manifest should now include the docs from
x86_64-apple-darwin. Although those docs are slightly different, it
should be close enough. When aarch64-apple-darwin heads towards tier 1,
we can revisit whether or not to re-enable the docs.
Exclude llvm-libunwind from the self-contained set on s390x-musl targets
llvm-libunwind does not support s390x targets at present, so we cannot build it
for s390x targets. Accordingly, remove it from the self-contained set.
