Make `TrustedStep` require `Copy`
All the implementations of the trait already are `Copy`, and this seems to be enough to simplify the implementations enough to make the MIR inliner willing to inline basics like `Range::next`.
r? `@thomcc`
Load only the crate header for `locator::crate_matches`
Previously, we used the following info to determine whether to load the crate:
1. The METADATA_HEADER, which includes a METADATA_VERSION constant
2. The embedded rustc version
3. Various metadata in the `CrateRoot`, including the SVH
This worked ok most of the time. Unfortunately, when building locally the rustc version is always
the same because `omit-git-hash` is on by default. That meant that we depended only on 1 and 3, and
we are not very good about bumping METADATA_VERSION (it's currently at 7) so in practice we were
only depending on 3. `CrateRoot` is a very large struct and changes somewhat regularly, so this led
to a steady stream of crashes from trying to load it.
Change the logic to add an intermediate step between 2 and 3: introduce a new `CrateHeader` struct
that contains only the minimum info needed to decide whether the crate should be loaded or not. That
avoids having to load all of `CrateRoot`, which in practice means we should crash much less often.
Note that this works because the SVH should be different between any two dependencies, even if the
compiler has changed, because we use `-Zbinary-dep-depinfo` in bootstrap. See
https://github.com/rust-lang/rust/pull/111329#issuecomment-1538303474 for more details about how the
original crash happened.
Use `Cow` in `{D,Subd}iagnosticMessage`.
Each of `{D,Subd}iagnosticMessage::{Str,Eager}` has a comment:
```
// FIXME(davidtwco): can a `Cow<'static, str>` be used here?
```
This commit answers that question in the affirmative. It's not the most compelling change ever, but it might be worth merging.
This requires changing the `impl<'a> From<&'a str>` impls to `impl From<&'static str>`, which involves a bunch of knock-on changes that require/result in call sites being a little more precise about exactly what kind of string they use to create errors, and not just `&str`. This will result in fewer unnecessary allocations, though this will not have any notable perf effects given that these are error paths.
Note that I was lazy within Clippy, using `to_string` in a few places to preserve the existing string imprecision. I could have used `impl Into<{D,Subd}iagnosticMessage>` in various places as is done in the compiler, but that would have required changes to *many* call sites (mostly changing `&format("...")` to `format!("...")`) which didn't seem worthwhile.
r? `@WaffleLapkin`
Each of `{D,Subd}iagnosticMessage::{Str,Eager}` has a comment:
```
// FIXME(davidtwco): can a `Cow<'static, str>` be used here?
```
This commit answers that question in the affirmative. It's not the most
compelling change ever, but it might be worth merging.
This requires changing the `impl<'a> From<&'a str>` impls to `impl
From<&'static str>`, which involves a bunch of knock-on changes that
require/result in call sites being a little more precise about exactly
what kind of string they use to create errors, and not just `&str`. This
will result in fewer unnecessary allocations, though this will not have
any notable perf effects given that these are error paths.
Note that I was lazy within Clippy, using `to_string` in a few places to
preserve the existing string imprecision. I could have used `impl
Into<{D,Subd}iagnosticMessage>` in various places as is done in the
compiler, but that would have required changes to *many* call sites
(mostly changing `&format("...")` to `format!("...")`) which didn't seem
worthwhile.
MIR: opt-in normalization of `BasicBlock` and `Local` numbering
This doesn't matter at all for actual codegen, but after spending some time reading pre-codegen MIR, I was wishing I didn't have to jump around so much in reading post-inlining code.
So this add two passes that are off by default for every mir level, but can be enabled (`-Zmir-enable-passes=+ReorderBasicBlocks,+ReorderLocals`) for humans.
Migrate `item_foreign_type` to Askama
This PR continues the migration of `print_item.rs` functions to Askama. This piece of work migrates the function `item_foreign_type`
Refers https://github.com/rust-lang/rust/issues/108868
Add support for LLVM SafeStack
Adds support for LLVM [SafeStack] which provides backward edge control
flow protection by separating the stack into two parts: data which is
only accessed in provable safe ways is allocated on the normal stack
(the "safe stack") and all other data is placed in a separate allocation
(the "unsafe stack").
SafeStack support is enabled by passing `-Zsanitizer=safestack`.
[SafeStack]: https://clang.llvm.org/docs/SafeStack.html
cc `@rcvalle` #39699
This reverts commit 9267843e72, reversing
changes made to e52fbff5e8.
This breaks our ability to bump the src/version where we're bootstrapping with an older compiler
than usual (according to version number). It's not clear whether the intended use case has a clean
solution given this constraint, so reverting for now - we can reland with a fix of some kind implemented.
Clean up usage of `cx.tcx` when `tcx` is already set into a variable
I discovered a few cases where `cx.tcx` (and equivalents) was used whereas `tcx` was already stored into a variable. In those cases, better to just use `tcx` directly.
r? `@notriddle`
rustc driver: Remove argument 0 before at-expansion to prevent ICE
Under Unix-based operating systems, when I execute rustc by setting argv0 to ``@/dev/null`,` it will expand command-line arguments from this file, leading to an empty arglist, which then triggers an ICE by trying to remove first argument.
The panic message is this:
```
thread 'main' panicked at 'range start index 1 out of range for slice of length 0', compiler/rustc_driver/src/lib.rs:972:17
```
My fix is to remove the first argument before expanding arguments.
<details>
<summary>Full backtrace</summary>
```sh
% (exec -a `@/dev/null` `rustup which rustc`)
thread 'main' panicked at 'range start index 1 out of range for slice of length 0', compiler/rustc_driver/src/lib.rs:972:17
stack backtrace:
0: 0x7fcec776659a - std::backtrace_rs::backtrace::libunwind::trace::h595f06c70adcc478
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x7fcec776659a - std::backtrace_rs::backtrace::trace_unsynchronized::h177a0149c76cdde9
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7fcec776659a - std::sys_common::backtrace::_print_fmt::hc0701fd2c3530c58
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/sys_common/backtrace.rs:65:5
3: 0x7fcec776659a - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hd4cd115d8750fd6c
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7fcec77c839e - core::fmt::write::h93e2f5923c7eca08
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/core/src/fmt/mod.rs:1213:17
5: 0x7fcec7756be5 - std::io::Write::write_fmt::h8162dbb45f0b9e62
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/io/mod.rs:1682:15
6: 0x7fcec7766365 - std::sys_common::backtrace::_print::h1835ef8a8f9066da
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/sys_common/backtrace.rs:47:5
7: 0x7fcec7766365 - std::sys_common::backtrace::print::hcb5e6388b9235f41
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/sys_common/backtrace.rs:34:9
8: 0x7fcec776912f - std::panicking::default_hook::{{closure}}::h9c084969ccf9a722
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:267:22
9: 0x7fcec7768e6b - std::panicking::default_hook::h68fa2ba3c3c6c12f
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:286:9
10: 0x7fcecaab56e4 - <rustc_driver[f4ad927b3c57833d]::DEFAULT_HOOK::{closure#0}::{closure#0} as core[d16e85342ea223d9]::ops::function::FnOnce<(&core[d16e85342ea223d9]::panic::panic_info::PanicInfo,)>>::call_once::{shim:vtable#0}
11: 0x7fcec776996a - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h4e6ced11e07d8b24
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/alloc/src/boxed.rs:2002:9
12: 0x7fcec776996a - std::panicking::rust_panic_with_hook::h8d5c434518ef298c
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:692:13
13: 0x7fcec77696e9 - std::panicking::begin_panic_handler::{{closure}}::hf33414f5dabf6faf
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:579:13
14: 0x7fcec7766a4c - std::sys_common::backtrace::__rust_end_short_backtrace::hc50389427413bb75
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/sys_common/backtrace.rs:137:18
15: 0x7fcec77693f2 - rust_begin_unwind
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:575:5
16: 0x7fcec77c4d43 - core::panicking::panic_fmt::h2de7a7938f816de8
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/core/src/panicking.rs:64:14
17: 0x7fcec77cb492 - core::slice::index::slice_start_index_len_fail_rt::h0c87d85ce11d10f6
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/core/src/slice/index.rs:53:5
18: 0x7fcec77cb416 - core::slice::index::slice_start_index_len_fail::h504609f2a6b168d1
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/core/src/slice/index.rs:41:9
19: 0x7fceca0eca1f - rustc_driver[f4ad927b3c57833d]::handle_options
20: 0x7fceca0e037f - <rustc_driver[f4ad927b3c57833d]::RunCompiler>::run
21: 0x7fceca0dfd0d - <core[d16e85342ea223d9]::panic::unwind_safe::AssertUnwindSafe<rustc_driver[f4ad927b3c57833d]::main::{closure#0}> as core[d16e85342ea223d9]::ops::function::FnOnce<()>>::call_once
22: 0x7fceca17ce89 - rustc_driver[f4ad927b3c57833d]::main
23: 0x564f5f008a87 - rustc_main[f164605d1302e295]::main
24: 0x564f5f008973 - std[3da461b304582a2c]::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
25: 0x564f5f008969 - <std[3da461b304582a2c]::rt::lang_start<()>::{closure#0} as core[d16e85342ea223d9]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
26: 0x7fcec774795c - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h699977d052768608
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/core/src/ops/function.rs:287:13
27: 0x7fcec774795c - std::panicking::try::do_call::h4e121e623c70f903
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:483:40
28: 0x7fcec774795c - std::panicking::try::hf9d919e062bc178a
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:447:19
29: 0x7fcec774795c - std::panic::catch_unwind::h7a7b12272684cb97
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panic.rs:140:14
30: 0x7fcec774795c - std::rt::lang_start_internal::{{closure}}::hd96b0eb4844b8762
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/rt.rs:148:48
31: 0x7fcec774795c - std::panicking::try::do_call::h1af1f88f4f92a22c
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:483:40
32: 0x7fcec774795c - std::panicking::try::hf20d7abea7f0f097
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panicking.rs:447:19
33: 0x7fcec774795c - std::panic::catch_unwind::hb0e084c3a9c042e4
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/panic.rs:140:14
34: 0x7fcec774795c - std::rt::lang_start_internal::hca9d5c7277f5b67c
at /rustc/2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74/library/std/src/rt.rs:148:20
35: 0x564f5f008ab7 - main
36: 0x7fcec74a1790 - <unknown>
37: 0x7fcec74a184a - __libc_start_main
38: 0x564f5f00899e - <unknown>
39: 0x0 - <unknown>
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
note: rustc 1.68.0 (2c8cc3432 2023-03-06) running on x86_64-unknown-linux-gnu
query stack during panic:
end of query stack
```
</details>
I also checked if I can trigger a similar problem by passing empty argument list to `execve`, but at least under Linux, it seems to always insert an empty first argument if there are none.
Rework handling of recursive panics
This PR makes 2 changes to how recursive panics works (a panic while handling a panic).
1. The panic count is no longer used to determine whether to force an immediate abort. This allows code like the following to work without aborting the process immediately:
```rust
struct Double;
impl Drop for Double {
fn drop(&mut self) {
// 2 panics are active at once, but this is fine since it is caught.
std::panic::catch_unwind(|| panic!("twice"));
}
}
let _d = Double;
panic!("once");
```
Rustc already generates appropriate code so that any exceptions escaping out of a `Drop` called in the unwind path will immediately abort the process.
2. Any panics while the panic hook is executing will force an immediate abort. This is necessary to avoid potential deadlocks like #110771 where a panic happens while holding the backtrace lock. We don't even try to print the panic message in this case since the panic may have been caused by `Display` impls.
Fixes#110771
Fix re-export of doc hidden macro not showing up
It's part of the follow-up of https://github.com/rust-lang/rust/pull/109697.
Re-exports of doc hidden macros should be visible. It was the only kind of re-export of doc hidden item that didn't show up.
r? `@notriddle`
Include test suite metadata in the build metrics
This PR enhances the build metadata to include structured information about the test suites being executed, allowing external tools consuming the metadata to understand what was being tested.
The included metadata is:
* Target triple
* Host triple
* Stage number
* For compiletest tests:
* Suite name
* Mode
* Comparing mode
* For crate tests:
* List of crate names
This is implemented by replacing the `test` JSON node with a new `test_suite` node, which contains the metadata and the list of tests. This change also improves the handling of multiple test suites executed in the same step (for example in compiletest tests with a compare mode), as the multiple test suite executions will now be tracked in separate `test_suite` nodes.
This included a breaking change in the build metrics metadata format. To better handle this, in the second commit this PR introduces the `metadata_version` top-level field. The old version is considered to be `0`, while the new one `1`. Bootstrap will also gracefully handle existing metadata of a different version.
r? `@jyn514`
Generate docs for bootstrap itself
This verifies the intra-doc links are correct, and hopefully makes things easier for new contributors.
Note that this will conflict with https://github.com/rust-lang/rust/pull/111955, i'm pretty sure i typo-ed some of the intra-doc links lol
Migrate `item_static` to Askama
This pull request addresses the type signature of the item_static function in our codebase. Previously, this function accepted a mutable reference to a Buffer for writing output. The current changes modify this to instead accept a mutable reference to any type that implements the Write trait.
Referes https://github.com/rust-lang/rust/issues/108868
Adds support for LLVM [SafeStack] which provides backward edge control
flow protection by separating the stack into two parts: data which is
only accessed in provable safe ways is allocated on the normal stack
(the "safe stack") and all other data is placed in a separate allocation
(the "unsafe stack").
SafeStack support is enabled by passing `-Zsanitizer=safestack`.
[SafeStack]: https://clang.llvm.org/docs/SafeStack.html
Improve startup time of bootstrap
~~If the user has a `build/host` symlink set up, we can determine the target triple by reading it rather than invoking rustc. This significantly reduces startup time of bootstrap once any kind of build has been done~~
New approach explained below
```
➜ hyperfine -p 'git checkout -q master' -N './x.py -h' -r 50
Benchmark 1: ./x.py -h
Time (mean ± σ): 140.7 ms ± 2.6 ms [User: 99.9 ms, System: 39.3 ms]
Range (min … max): 136.8 ms … 149.6 ms 50 runs
➜ rust git:(master) hyperfine -p 'git checkout -q speedup-bootstrap-py' -N './x.py -h' -r 50
Benchmark 1: ./x.py -h
Time (mean ± σ): 95.2 ms ± 1.5 ms [User: 67.7 ms, System: 26.7 ms]
Range (min … max): 92.9 ms … 99.6 ms 50 runs
```
Also a small microoptimisation in using string splitting rather than regex when reading toml, which saves a few more milliseconds (2-5 testing locally), but less important.
Profiling shows the remaining runtime is around half setting up the Python runtime, and the vast majority of the remaining time is spent in subprocess building and running bootstrap itself, so probably can't be improved much further.
Ensure Fluent messages are in alphabetical order
Fixes#111847
This adds a tidy check to ensure Fluent messages are in alphabetical order, as well as sorting all existing messages. I think the error could be worded better, would appreciate suggestions.
<details>
<summary>Script used to sort files</summary>
```py
import sys
import re
fn = sys.argv[1]
with open(fn, 'r') as f:
data = f.read().split("\n")
chunks = []
cur = ""
for line in data:
if re.match(r"^([a-zA-Z0-9_]+)\s*=\s*", line):
chunks.append(cur)
cur = ""
cur += line + "\n"
chunks.append(cur)
chunks.sort()
with open(fn, 'w') as f:
f.write(''.join(chunks).strip("\n\n") + "\n")
```
</details>
