Since its inception rustbuild has always worked in three stages: one for
libstd, one for libtest, and one for rustc. These three stages were
architected around crates.io dependencies, where rustc wants to depend
on crates.io crates but said crates don't explicitly depend on libstd,
requiring a sysroot assembly step in the middle. This same logic was
applied for libtest where libtest wants to depend on crates.io crates
(`getopts`) but `getopts` didn't say that it depended on std, so it
needed `std` built ahead of time.
Lots of time has passed since the inception of rustbuild, however,
and we've since gotten to the point where even `std` itself is depending
on crates.io crates (albeit with some wonky configuration). This
commit applies the same logic to the two dependencies that the `test`
crate pulls in from crates.io, `getopts` and `unicode-width`. Over the
many years since rustbuild's inception `unicode-width` was the only
dependency picked up by the `test` crate, so the extra configuration
necessary to get crates building in this crate graph is unlikely to be
too much of a burden on developers.
After this patch it means that there are now only two build phasese of
rustbuild, one for libstd and one for rustc. The libtest/libproc_macro
build phase is all lumped into one now with `std`.
This was originally motivated by rust-lang/cargo#7216 where Cargo was
having to deal with synthesizing dependency edges but this commit makes
them explicit in this repository.
Run Clippy without json-rendered flag
Removed in https://github.com/rust-lang/rust/pull/62766
Replacing it with `--json=diagnostic-rendered-ansi` fails:
```
error: using `--json` requires also using `--error-format=json`
```
Running `./x.py clippy src/libstd` locally works fine (with colors) on Linux so I don't know if there is something to fix.
Do not emit JSON dumps of diagnostic codes
This decouples the error index generator from libsyntax for the most part (though it still depends on librustdoc for the markdown parsing and generation).
Fixes#34588
This is no longer used by the index generator and was always an unstable
compiler detail, so strip it out.
This also leaves in RUSTC_ERROR_METADATA_DST since the stage0 compiler
still needs it to be set.
This patch supports less behavior than before, since specifiying stage 1
vs stage 2 is no longer possible, but that is presumably a somewhat rare
use case anyway, so not supporting it seems acceptable (and it can be
readded easily if desired).
Opaque builtin derive macros
* Buiilt-in derives are now opaque macros
* This required limiting the visibility of some previously unexposed functions in `core`.
* This also required the change to `Ident` serialization.
* All gensyms are replaced with hygienic identifiers
* Use hygiene to avoid most other name-resolution issues with buiilt-in derives.
* As far as I know the only remaining case that breaks is an ADT that has the same name as one of its parameters. Fixing this completely seemed to be more effort than it's worth.
* Remove gensym in `Ident::decode`, which lead to linker errors due to `inline` being gensymmed.
* `Ident`now panics if incremental compilation tries to serialize it (it currently doesn't).
* `Ident` no longer uses `gensym` to emulate cross-crate hygiene. It only applied to reexports.
* `SyntaxContext` is no longer serializable.
* The long-term fix for this is to properly implement cross-crate hygiene, but this seemed to be acceptable for now.
* Move type/const parameter shadowing checks to `resolve`
* This was previously split between resolve and type checking. The type checking pass compared `InternedString`s, not Identifiers.
* Removed the `SyntaxContext` from `{ast, hir}::{InlineAsm, GlobalAsm}`
cc #60869
r? @petrochenkov
Modify librustc_llvm to pass -DNDEBUG while compiling.
Currently, librustc_llvm builds are not reproducible because the LLVM files it compiles use the debug version of llvm_unreachable, which uses __FILE__. To fix this, we propagate NDEBUG from bootstrap if applicable and use it when compiling librustc_llvm.
r? @alexcrichton
Rename overflowing_{add,sub,mul} intrinsics to wrapping_{add,sub,mul}.
These confused @Gankra, and then, also me, especially since `overflowing_*` *methods* also exist, but they map to `*_with_overflow` intrinsics!
r? @oli-obk / @nikomatsakis cc @Mark-Simulacrum (on the rustbuild workaround)
Remap paths for proc-macro crates.
The remap-debuginfo config option remaps paths in most crates, but it does not apply to proc-macros, so they are still non-reproducible. This patch fixes that.
I'm not completely sure if this is the best way to do this, but to get reproducible builds we need librustc_macros to be built with --remap-path-prefix. I was previously modifying Cargo to pass that argument to all child crates, so this seems simpler and more correct.
I did not add a test since there do not seem to be any existing tests for RUSTC_DEBUGINFO_MAP.
r? @alexcrichton
Rollup of 10 pull requests
Successful merges:
- #60492 (Add custom nth_back for Chain)
- #61780 (Finalize the error type for `try_reserve`)
- #63495 ( Remove redundant `ty` fields from `mir::Constant` and `hair::pattern::PatternRange`.)
- #63525 (Make sure that all file loading happens via SourceMap)
- #63595 (add sparc64-unknown-openbsd target)
- #63604 (Some update for vxWorks)
- #63613 (Hygienize use of built-in macros in the standard library)
- #63632 (A couple of comment fixes.)
- #63634 (ci: properly set the job name in CPU stats)
- #63636 (ci: move linkcheck from mingw-2 to mingw-1)
Failed merges:
r? @ghost
ci: move linkcheck from mingw-2 to mingw-1
Running UI tests now takes a huge amount of time on mingw builders
(between 40 and 50 minutes), with mingw-1 builders taking even an hour
less to finish than mingw-2. This PR moves linkcheck from mingw-2 to
mingw-1, removing between 10 and 20 minutes of runtime on the -2
builders.
r? @alexcrichton
Some update for vxWorks
1. support crt-static
2. change armv7_wrs_vxworks to armv7_wrs_vxworks_eabihf.
3. change vx-cxx to wr-c++, vx-ar to wr-ar and vx-run to wr-run.
4. code cleanup
r? @alexcrichton
Running UI tests now takes a huge amount of time on mingw builders
(between 40 and 50 minutes), with mingw-1 builders taking even an hour
less to finish than mingw-2. This PR moves linkcheck from mingw-2 to
mingw-1, removing between 10 and 20 minutes of runtime on the -2
builders.
Utilize -Zbinary-dep-depinfo in rustbuild
We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.
Both of these are fixable in the longer term but this existing patch gives us the following benefits:
* We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes#54712.
* Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes#54008, fixes#50481.
* We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes#53284, and fixes#52719.
* Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes#49979 (but no reproduction steps in that issue). Fixes#59105.
cc #63012
Currently, librustc_llvm builds are not reproducible because the LLVM
files it compiles use the debug version of llvm_unreachable, which
uses __FILE__. To fix this, we propagate NDEBUG from bootstrap if
applicable and use it when compiling librustc_llvm.
The remap-debuginfo config option remaps paths in most crates, but it
does not apply to proc-macros, so they are still non-reproducible.
This patch fixes that.
Fix rustc-guide toolstate tracking.
The rustc-guide is still not saving its toolstate. It needs to be done explicitly.
Note: There are some confusing bits here, like without `--no-fail-fast`, it saves the wrong value in `toolstates.json`. Also, "rustbook" is always added as "test-fail". Presumably [this code](https://github.com/rust-lang/rust/blame/63c1f17d950bfefc8f356777f124f304f0575c76/src/bootstrap/tool.rs#L201-L205) assumes everything fails until it passes tests, which is a bit confusing for things that don't run tests.
cc @mark-i-m
rustbuild: RISC-V is no longer an experimental LLVM target
This moves RISC-V from the experimental LLVM targets to the
regular LLVM targets. RISC-V was made non-experimental in
https://reviews.llvm.org/rL366399
I have also sorted the list of LLVM targets, and changed the code
around setting llvm_exp_targets (and its default) to match the code
setting llvm_targets (and its default), ensuring future changes to
the defaults, as LLVM targets become stable, affect as few places as
possible.
Given WebAssembly is in `LLVM_ALL_TARGETS` and is therefore built by default (and has been since October 2018), I'm not sure why rust still has it in `experimental-targets`. I'm happy to update this PR to move it into the main list of LLVM targets.
r? @alexcrichton
build_helper: try less confusing method names
build_helper's `*_silent` methods were likely called that way because they do not print the command being run to stdout. [In the original file this all makes sense](046e6874c4 (diff-5c3d6537a43ecae03014e118a7fe3321)). But later it also gained `*_suppressed` methods and the difference between `silent` and `suppressed` is far from clear.
So rename `run` (which prints the command being run) to `run_verbose`. Then we can call the methods that just run a command and show its output but nothing extra `run` and `try_run`.
`run_verbose` (formerly `run`) is unused from what I can tell. Should I remove it?
r? @alexcrichton
Cc @Mark-Simulacrum
Also see https://github.com/rust-lang/rust/pull/63089#discussion_r308018890.
Added support for armv7-unknown-linux-gnueabi/musleabi
Fixes#63101
Some things that are not done and I hope someone can help me with:
* During the ci build of `armv7-unknown-linux-gnueabi` `openssl` must be built (to build cargo) but `openssl` does not yet support this target. This feels slightly like a chicken-and-egg problem, any feedback is welcome.
* Should I add any tests for any of these targets?
