bootstrap: extract builder cargo to its own module
I was looking at our cargo rustflags/rustdocflags usages, and I found `builder.rs` to be a large
file which made it hard to digest. This PR tries to break out the cargo command wrapper parts to
its own submodule to make it easier to identify builder cargo-specific logic.
This PR:
- Extracts the cargo command wrapper to its own module and also move `Builder::{bare_,}cargo` impl
to the submodule.
- Reorganizes some imports in `lib.rs` (no functional changes).
- Slightly adjusts some docs in `builder.rs`.
This PR is basically just moving code around, and should not contain any functional changes.
Before this PR, `builder.rs` was 2743 lines. After this PR, `builder.rs` is down to a more
manageable 1386 lines and `cargo.rs` is 1085 lines.
I found builder.rs to be a massive file which made it hard to digest. To
make `RUSTFLAGS` usage hardening easier later, I extracted the cargo
part in `builder.rs` into its own module.
ci update freebsd version proposal, freebsd 12 being eol
raising to the lowest still active supported freebsd version.
From 13.1 (already eol too), freebsd introduces a cpu affinity layer
with linux. It also introduces a api compatible copy_file_range which
can be used like its linux's counterpart.
The former is essential to build https://github.com/rust-lang/rust/pull/120589, therefore breaks the backward
compatibility with the previous FreeBSD releases.
Blocked on https://github.com/rust-lang/rust/issues/130465
feat(rustdoc-json-types): mark simple enums as copy
Fixesrust-lang/rustdoc-types#26 and some typos in the documentation
r? `@aDotInTheVoid`
I have been assigning these PRs to you `@aDotInTheVoid,` is that okay? I think I'm out of PRs for now, but for future reference c:
fix(rustdoc-json-types): document rustc-hash feature
The `rustc-hash` feature is publicly exposed by the `rustdoc-types`. It is already documented in that crate's README and Cargo.toml, but we might as well add some information to the crate docs themselves c:
Follow up to:
- #131936
- [rust-lang/rustdoc-types#42][1]
[1]: https://github.com/rust-lang/rustdoc-types/pull/42
r? `@aDotInTheVoid`
compiletest: tidy up how `tidy` and `tidy` (html version) are disambiguated
Rename `has_tidy` -> `has_html_tidy` (`tidy` is also a bootstrap tool, but rustdoc uses a html tidy that has the same binary name). Follow-up to #131941.
Also apparently `runtest.rs` is short enough now, we can delete the `tidy` (bootstrap version) ignore for file length.
Align boolean option descriptions in `configure.py`
Boolean options are currently printed as
```
Options
--enable-debug OR --disable-debug enables debugging environment; does not affect optimization of bootstrapped code
--enable-docs OR --disable-docs build standard library documentation
--enable-compiler-docs OR --disable-compiler-docs build compiler documentation
--enable-optimize-tests OR --disable-optimize-tests build tests with optimizations
--enable-verbose-tests OR --disable-verbose-tests enable verbose output when running tests
--enable-ccache OR --disable-ccache invoke gcc/clang via ccache to reuse object files between builds
--enable-sccache OR --disable-sccache invoke gcc/clang via sccache to reuse object files between builds
--enable-local-rust OR --disable-local-rust use an installed rustc rather than downloading a snapshot
--local-rust-root=VAL set prefix for local rust binary
--enable-local-rebuild OR --disable-local-rebuild assume local-rust matches the current version, for rebuilds; implies local-rust, and is implied if local-rust already matches the current version
```
as of #131117
imo, this is a little difficult to skim. This PR changes this to align the `OR`s and push the description onto a newline:
```
Options
--enable-debug OR --disable-debug
enables debugging environment; does not affect optimization of bootstrapped code
--enable-docs OR --disable-docs
build standard library documentation
--enable-compiler-docs OR --disable-compiler-docs
build compiler documentation
--enable-optimize-tests OR --disable-optimize-tests
build tests with optimizations
--enable-verbose-tests OR --disable-verbose-tests
enable verbose output when running tests
--enable-ccache OR --disable-ccache
invoke gcc/clang via ccache to reuse object files between builds
--enable-sccache OR --disable-sccache
invoke gcc/clang via sccache to reuse object files between builds
--enable-local-rust OR --disable-local-rust
use an installed rustc rather than downloading a snapshot
--local-rust-root=VAL set prefix for local rust binary
--enable-local-rebuild OR --disable-local-rebuild
assume local-rust matches the current version, for rebuilds; implies local-rust, and is implied if local-rust already matches the current version
```
Register `src/tools/unicode-table-generator` as a runnable tool
It seems like `src/tools/unicode-table-generator` is not currently managed by bootstrap. This PR wires it up with bootstrap as a runnable tool.
This tool seems to take two possible args:
1. (Mandatory) path to `library/core/src/unicode/unicode_data.rs`, and
2. (Optional) path to generate a test file.
I only passed the mandatory path to `unicode_data.rs` in bootstrap and didn't do anything about (2). I'm not sure about how this tool is supposed to be run.
`Cargo.lock` is modified because I renamed `unicode-table-generator`'s bin name to match the tool name, as bootstrap's tool running logic expects the bin name to be derived from the tool name.
I also added a triagebot message to remind to not manually edit the library source file and edit the tool then regenerate instead, but this should probably be a tidy check (if that's desirable then that can be in a follow-up PR, though may be overkill).
Helps with #131640 but does not close it because still no docs.
r? `@Mark-Simulacrum` (since I think you authored this tool?)
Fix missing rustfmt in msi installer #101993
# Context
- Fixed missing `rustfmt`, `clippy`, `miri` and `rust-analyzer` in msi installer
- Fixed missing `rustfmt` for apple darwin installer
- Closes#101993
r? `@jyn514`
- Please let me know if I should request from someone else instead. I divided the changes into 3 separate commits for the ease of review. The refactoring commit `fbdfd5c03c3c979bcf105ccdd05ff4ab9f37a763` is a bit more involved, but I think it helps in the long term for readability and to avoid bugs.
- I changed `build-manifest` to `build_manifest` in order to invoke it as a library. Not sure if this is gonna break any upstream processes. I checked `generate-manifest-list` and `generate-release` but didn't find any obvious reference
- Will push fixes for linting later
Allow `#[deny]` inside `#[forbid]` as a no-op
Forbid cannot be overriden. When someome tries to do this anyways, it results in a hard error. That makes sense.
Except it doesn't, because macros. Macros may reasonably use `#[deny]` (or `#[warn]` for an allow-by-default lint) in their expansion to assert that their expanded code follows the lint. This is doesn't work when the output gets expanded into a `forbid()` context. This is pretty silly, since both the macros and the code agree on the lint!
By making it a warning instead, we remove the problem with the macro, which is now nothing as warnings are suppressed in macro expanded code, while still telling users that something is up.
fixes#121483
rustdoc: Clean up footnote handling
Best reviewed commit by commit.
Extracts footnote handling logic into it's own file (first commit) and then makes that file slightly nicer to read/understand.
No functional changes, but lays the groundwork for making more changes to footnotes (eg #131901, #131946)
compiler: Adopt rust-analyzer impls for `LayoutCalculatorError`
We're about to massively churn the internals of `rustc_abi`. To minimize the immediate and future impact on rust-analyzer, as a subtree that depends on this crate, grow some API on `LayoutCalculatorError` that reflects their uses of it. This way we can nest the type in theirs, and they can just call functions on it without having to inspect and flatten-out its innards.
compiletest: disambiguate html-tidy from rust tidy tool
when i first saw this error message i was very confused, i thought it was talking about `src/tools/tidy`. now it should be much more clear what tool should be installed.
feat(rustdoc-json-types): introduce rustc-hash feature
This allows the public `rustdoc-types` crate to expose this feature easily and allows consumers of the crate to get the performance advantages from doing so.
The reasoning for this was discussed on [Zulip][1]
Changes:
- Make `rustc-hash` optional but default to including it
- Rename all occurrences of `FxHashMap` to `HashMap`.
- Feature gate the import and rename the imported `FxHashMap` to `HashMap`
- Introduce a type alias `FxHashMap` which resolves to the currently used `HashMap` (`rustc_hash::FxHashMap` or `std::collections::HashMap`) for use in `src/librustdoc`.
[1]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/rustc-hash.20and.20performance.20of.20rustdoc-types
**extra context from the zulip thread:**
- `@obi1kenobi` requested benchmarks of the switch to `rustc-hash`
- I benchmarked switching `rustdoc-types` to `rustc-hash` which yielded a ~300ms improvement to `cargo-semver-checks`'s index building step (this step is done twice so the improvements are ~150ms per index).
- The benchmarks were presented in Zulip and people were in favor of introducing `rustc-hash` to the public `rustdoc-types` crate.
- There were differing opinions on how to introduce the dependency:
1. "Hard" dependency: remove use of `std::collections::HashMap` in favor of `FxHashMap`.
2. "Soft" dependency: make optional and introduce a feature then enable/disable it by default (this PR).
3. ~~Make `rustdoc-types` generic and expose the `RandomState`~~ (a lot of work & complexity for little gain over a feature gate).
`@obi1kenobi` and I prefer the feature gate so that is what I am adding here.
My reasons for the preference are:
- `cargo-semver-checks` is especially perf sensitive, we don't expect people to care about ~150ms extra time when reading in a 500MB file (the size of the sample we used for benchmarking).
- Keeping `rustdoc-types` lean by having its only direct dependency be `serde` is nice for the general consumer of the crate.
- `rustc-hash` is not HashDOS resistant (but it is questionable whether `rustdoc-types` would be used on adversarial inputs).
r? `@aDotInTheVoid`
This allows the public `rustdoc-types` crate to expose this feature
easily and allows consumers of the crate to get the performance
advantages from doing so.
The reasoning for this was discussed on [Zulip][1]
Changes:
- Make `rustc-hash` optional but default to including it
- Rename all occurrences of `FxHashMap` to `HashMap`.
- Feature gate the import and rename the imported `FxHashMap` to
`HashMap`
- Introduce a type alias `FxHashMap` which resolves to the currently
used `HashMap` (`rustc_hash::FxHashMap` or
`std::collections::HashMap`) for use in `src/librustdoc`.
[1]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/rustc-hash.20and.20performance.20of.20rustdoc-types
small interpreter error cleanup
- Add `InterpretResult::map_err_kind` for the common case of swapping out the error kind (while preserving the backtrace pointing to the original error source)
- Rename `InterpError` -> `InterpErrorKind` to be consistent with the `kind` field name, and make it more clear that this is not the final error type
rustdoc: Switch from FxHash to sha256 for static file hashing.
Fixes https://github.com/rust-lang/rust/pull/129533#issuecomment-2422891519
fxhash isn't well defined, and it's implementation is being changed in #129533. But because rustdoc uses it for static files (and encodes that hashing in rustdoc.css), this broke our tests. Given that this isn't performace critical, I think the right fix is to used a well-defined hash that will never change its definition. I've picked (rather arbitrarily) sha256.
bootstrap: allow setting `--jobs` in config.toml
Allow setting `--jobs` in config.toml's `[build]` section.
```toml
[build]
jobs = 0
```
If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std:🧵:available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up.
Closes#131836.
r? bootstrap
Update cargo
7 commits in 8c30ce53688e25f7e9d860b33cc914fb2957ca9a..cf53cc54bb593b5ec3dc2be4b1702f50c36d24d5
2024-10-15 16:43:16 +0000 to 2024-10-18 13:56:15 +0000
- feat: Stabilize MSRV-aware resolver config (rust-lang/cargo#14639)
- Help with `[patch.crates.io]` (rust-lang/cargo#14700)
- test: Migrate publish snapshotting to snapbox (rust-lang/cargo#14642)
- Bump to 0.85.0; update changelog (rust-lang/cargo#14695)
- Fix typo in faq.md (rust-lang/cargo#14696)
- fix(registry): `HttpRegistry` `block_until_ready` returns early when work is still pending (rust-lang/cargo#14694)
- fix(resolver): avoid cloning when iterating using RcVecIter (rust-lang/cargo#14690)
Clippy subtree update
One day late with the sync, as I was sick yesterday.
Cargo.lock update includes Clippy version bump and some deps cleanup we did in Clippy to match more versions used in the Rust repo.
r? `@Manishearth`
Forbid cannot be overriden. When someome tries to do this anyways,
it results in a hard error. That makes sense.
Except it doesn't, because macros. Macros may reasonably use `#[deny]`
in their expansion to assert
that their expanded code follows the lint. This is doesn't work when the
output gets expanded into a `forbid()` context. This is pretty silly,
since both the macros and the code agree on the lint!
Therefore, we allow `#[deny(..)]`ing a lint that's already forbidden,
keeping the level at forbid.
compiletest: Store test collection context/state in two structs
This is another incremental cleanup that untangles some of the parameter passing during test collection, making it easier to see which pieces of context information are read-only, and making it easier to find where each field is used.
Rollup of 9 pull requests
Successful merges:
- #130136 (Partially stabilize const_pin)
- #131755 (Regression test for AVR `rjmp` offset)
- #131774 (Add getentropy for RTEMS)
- #131802 (Dont ICE when computing coverage of synthetic async closure body)
- #131809 (Fix predicate signatures in retain_mut docs)
- #131858 (Remove outdated documentation for `repeat_n`)
- #131866 (Avoid use imports in `thread_local_inner!`)
- #131874 (Default to the medium code model on OpenHarmony LoongArch target)
- #131877 (checktools.sh: add link to issue for more context about disabled Miri tests)
r? `@ghost`
`@rustbot` modify labels: rollup
