Copy stage0 `rustc` binaries to `stage0-sysroot`
This is basically a revival of https://github.com/rust-lang/rust/pull/101711 and https://github.com/rust-lang/rust/pull/107956, with an added check that the full sysroot will only be created if the original rustc comes from `stage0/bin`.
What is/should be tested:
- [x] `rustup toolchain link stage0` (new libstd is used correctly)
- [x] `python3 x.py fmt dist --stage 0`
- [x] Custom rustc/cargo in `config.toml` (in this case this logic is ignored)
- [x] Perfbot (try perf run has succeeded)
- [x] Real use case (https://github.com/rust-lang/backtrace-rs/pull/542)
(Hopefully) fixes: https://github.com/rust-lang/rust/issues/101691
This is not the "end all, be all" solution to this problem, but as long as it resolves the basic use-case, and doesn't break perfbot, I say ship it. This code will probably be nuked anyway Soon™ because of the stage redesign.
Rollup of 7 pull requests
Successful merges:
- #112931 (Enable zlib in LLVM on aarch64-apple-darwin)
- #113158 (tests: unset `RUSTC_LOG_COLOR` in a test)
- #113173 (CI: include workflow name in concurrency group)
- #113335 (Reveal opaques in new solver)
- #113390 (CGU formation tweaks)
- #113399 (Structurally normalize again for byte string lit pat checking)
- #113412 (Add basic types to SMIR)
r? `@ghost`
`@rustbot` modify labels: rollup
Enable zlib in LLVM on aarch64-apple-darwin
Works on macOS 13.4, Xcode version 14.3.1.0.1.1683849156
This was disabled in #75500 on Apple Silicon Developer Transition Kit, but Apple appears to have fixed their zlib now
Fix submodule handling when the current branch is named after a tag
If:
1. The current branch has the same name as git tag, and
2. The current branch is set to track a remote other than `origin`, and
3. We try to update a submodule
then we'll get the following error:
```
; x c
Updating submodule src/doc/reference
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: 'personal' does not appear to be a git repository
fatal: Could not read from remote repository.
```
The problem is that 1. causes `git symbolic-ref --short HEAD` to try and disambiguate the branch from the tag using `heads/branch-name`, which breaks a previous workaround for a bug in `git submodule update` that uses the wrong remote.
Adapt the workaround to strip `heads/` from the output.
If:
1. The current branch has the same name as git tag, and
2. The current branch is set to track a remote other than `origin`, and
3. We try to update a submodule
then we'll get the following error:
```
; x c
Updating submodule src/doc/reference
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: 'personal' does not appear to be a git repository
fatal: Could not read from remote repository.
```
The problem is that 1. causes `git symbolic-ref --short HEAD` to try and disambiguate the branch
from the tag using `heads/branch-name`, which breaks a previous workaround for a bug in `git
submodule update` that uses the wrong remote.
Adapt the workaround to strip `heads/` from the output.
rust-installer & rls: remove exclusion from rustfmt & tidy
<strike>based on #112884</strike>
`rust-installer` and `rls` no longer submodules, but not removed from exclude list for rustfmt and tidy, preventing running fmt and lints on them.
Add simple markdown formatting to `rustc --explain` output
This is a second attempt at #104540, which is #63128 without dependencies.
This PR adds basic markdown formatting to `rustc --explain` output when available. Currently, the output just displays raw markdown: this works of course, but it really doesn't look very elegant. (output is `rustc --explain E0038`)
<img width="583" alt="image" src="https://github.com/rust-lang/rust/assets/13724985/ea418117-47af-455b-83c0-6fc59276efee">
After this patch, sample output from the same file:
<img width="693" alt="image" src="https://github.com/rust-lang/rust/assets/13724985/12f7bf9b-a3fe-4104-b74b-c3e5227f3de9">
This also obeys the `--color always/auto/never` command option. Behavior:
- If pager is available and supports color, print with formatting to the pager
- If pager is not available or fails print with formatting to stdout - otherwise without formatting
- Follow `--color always/never` if suppied
- If everything fails, just print plain text to stdout
r? `@oli-obk`
cc `@estebank`
(since the two of you were involved in the previous discussion)
Currently, the output of `rustc --explain foo` displays the raw markdown in a
pager. This is acceptable, but using actual formatting makes it easier to
understand.
This patch consists of three major components:
1. A markdown parser. This is an extremely simple non-backtracking recursive
implementation that requires normalization of the final token stream
2. A utility to write the token stream to an output buffer
3. Configuration within rustc_driver_impl to invoke this combination for
`--explain`. Like the current implementation, it first attempts to print to
a pager with a fallback colorized terminal, and standard print as a last
resort.
If color is disabled, or if the output does not support it, or if printing
with color fails, it will write the raw markdown (which matches current
behavior).
Pagers known to support color are: `less` (with `-r`), `bat` (aka `catbat`),
and `delta`.
The markdown parser does not support the entire markdown specification, but
should support the following with reasonable accuracy:
- Headings, including formatting
- Comments
- Code, inline and fenced block (no indented block)
- Strong, emphasis, and strikethrough formatted text
- Links, anchor, inline, and reference-style
- Horizontal rules
- Unordered and ordered list items, including formatting
This parser and writer should be reusable by other systems if ever needed.
Don't pass --test-args to `python -m unitest`
The args for unittest and cargo test are mutually incompatible. Suggest that people use `python -m unittest ...` manually instead.
This also changes `bootstrap_test.py` to be easier to run standalone; see the commit for details.
r? `@clubby789` cc https://github.com/rust-lang/rust/pull/112281#discussion_r1248849172
The args for unittest and cargo test are mutually incompatible. Suggest that people use `python -m unittest ...` manually instead.
This also changes `bootstrap_test.py` to be easier to run standalone; see the commit for details.
Set `channel = nightly` in dist profile
This avoids some channel-specific defaults leaking into local installs. It also makes it easier to set options for compiler/library/codegen profiles in the future, since they can be gated off `channel` instead of being duplicated between all three files.
Here are the exact things `channel` controls today:
68d458bb40/src/bootstrap/llvm.rs (L466-L470)85c4ea0138/src/bootstrap/config.rs (L1374-L1375)85c4ea0138/src/bootstrap/config.rs (L1464-L1465)
``@cuviper`` i expect you don't want any of those to be set in distro builds, right?
Convert `run-make/coverage-reports` tests to use a custom compiletest mode
I was frustrated by the fact that most of the coverage tests are glued together with makefiles and shell scripts, so I tried my hand at converting most of them over to a newly-implemented `run-coverage` mode/suite in compiletest.
This ~~*mostly*~~ resolves#85009, ~~though I've left a small number of the existing tests as-is because they would require more work to fix/support~~.
---
I had time to go back and add support for the more troublesome tests that I had initially skipped over, so this PR now manages to completely get rid of `run-make/coverage-reports`.
---
The patches are arranged as follows:
- Declare the new mode/suite in bootstrap
- Small changes to compiletest that will be used by the new mode
- Implement the new mode in compiletest
- Migrate most of the tests over
- Add more code to bootstrap and compiletest to support the remaining tests
- Migrate the remaining tests (with some temporary hacks to avoid re-blessing them)
- Remove the temporary hacks and re-bless the migrated tests
- Remove the unused remnants of `run-make/coverage-reports`
Switch some more Steps to `builder.msg`
In particular, this hides compiletest suites under a dropdown - some builders are configured with `verbose-tests`, which makes the suites take up many thousands of lines.
before:
```
Documenting stage1 cargo (x86_64-unknown-linux-gnu)
Check compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
```
after:
```
::group::Documenting stage1 cargo (x86_64-unknown-linux-gnu)
::group::Testing stage1 compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu)
```
found this from https://github.com/rust-lang-ci/rust/actions/runs/5366993848/jobs/9736792611
r? `@oli-obk`
Test the cargo args generated by bootstrap.py
I recommend reviewing this commit-by-commit using the instructions in https://rustc-dev-guide.rust-lang.org/git.html#moving-large-sections-of-code.
- Test cargo arguments passed by bootstrap.py
This moves a lot of code around, but the logic itself is not too terribly complicated.
- Move almost all logic in `def bootstrap` to the `RustBuild` class, to avoid mixing setting configuration with running commands
- Update various doctests to the new (more complete) RustBuild config. In particular, don't pretend that `bin_root` supports `build` being unset.
- Change `parse_args` not to use a global, to allow testing it
- Set BUILD_DIR appropriately so bootstrap.py doesn't panic because cargo isn't found
- Allow passing arguments to `bootstrap_test.py`
Previous, it used the built-in test runner, which doesn't support options unless they're manually passed in the script.
- Fix progress messages for configure in bootstrap_test.py
Before it would unconditionally print `configure-args = []`.
r? `@albertlarsan68` cc https://github.com/rust-lang/rust/pull/112089https://github.com/rust-lang/rust/pull/111979#issuecomment-1568525699
In particular, this hides compiletest suites under a dropdown - some builders are configured with `verbose-tests`, which makes the suites take up many thousands of lines.
before:
```
Documenting stage1 cargo (x86_64-unknown-linux-gnu)
Check compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
```
after:
```
::group::Documenting stage1 cargo (x86_64-unknown-linux-gnu)
::group::Testing stage1 compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu)
```
The old code actually replaced `.json` with `.bak` (so, `settings.bak`),
rather than appending `.bak` as claimed (`settings.json.bak`).
`Path::set_extension` can instead be used with dots:
> The new extension may contain dots and will be used in its entirety,
> but only the part after the final dot will be reflected in
> self.extension.
