Always use the Rust version in package names
The format of the tarballs produced by CI is roughly the following:
{component}-{release}-{target}.{ext}
While on the beta and nightly channels `{release}` is just the channel name, on the stable channel is either the Rust version or the version of the component we're shipping:
cargo-0.47.0-x86_64-unknown-linux-gnu.tar.xz
clippy-0.0.212-x86_64-unknown-linux-gnu.tar.xz
llvm-tools-1.46.0-x86_64-unknown-linux-gnu.tar.xz
miri-0.1.0-x86_64-unknown-linux-gnu.tar.xz
rls-1.41.0-x86_64-unknown-linux-gnu.tar.xz
rust-1.46.0-x86_64-unknown-linux-gnu.tar.xz
...
This makes it really hard to get the package URL without having access to the manifest (and there is no manifest on ci-artifacts.rlo), as there is no consistent version number to use.
This PR addresses the problem by always using the Rust version number as `{release}` for the stable channel, regardless of the version number of the component we're shipping. I chose that instead of "stable" to avoid breaking the URL scheme *that* much.
Rustup should not be affected by this change, as it fetches the URLs from the manifest. Unfortunately we don't have a way to test other clients before making a stable release, as this change only affects the stable channel.
r? `@Mark-Simulacrum`
The format of the tarballs produced by CI is roughly the following:
{component}-{release}-{target}.{ext}
While on the beta and nightly channels `{release}` is just the channel
name, on the stable channel is either the Rust version or the version of
the component we're shipping:
cargo-0.47.0-x86_64-unknown-linux-gnu.tar.xz
clippy-0.0.212-x86_64-unknown-linux-gnu.tar.xz
llvm-tools-1.46.0-x86_64-unknown-linux-gnu.tar.xz
miri-0.1.0-x86_64-unknown-linux-gnu.tar.xz
rls-1.41.0-x86_64-unknown-linux-gnu.tar.xz
rust-1.46.0-x86_64-unknown-linux-gnu.tar.xz
...
This makes it really hard to get the package URL without having access
to the manifest (and there is no manifest on ci-artifacts.rlo), as there
is no consistent version number to use.
This commit addresses the problem by always using the Rust version
number as `{release}` for the stable channel, regardless of the version
number of the component we're shipping. I chose that instead of "stable"
to avoid breaking the URL scheme *that* much.
Rustup should not be affected by this change, as it fetches the URLs
from the manifest. Unfortunately we don't have a way to test other
clients before making a stable release, as this change only affects the
stable channel.
Fix LitKind's byte buffer to use refcounted slice
While working on adding a new lint for clippy (see https://github.com/rust-lang/rust-clippy/pull/6044) for avoiding shared ownership of "mutable buffer" types (such as using `Rc<Vec<T>>` instead of `Rc<[T]>`), I noticed a type exported from rustc_ast and used by clippy gets caught by the lint. This PR fixes the exported type.
This PR includes the actual change to clippy too, but I will open a PR directly against clippy for that part (although it will currently fail to build there).
This is a combination of 18 commits.
Commit #2:
Additional examples and some small improvements.
Commit #3:
fixed mir-opt non-mir extensions and spanview title elements
Corrected a fairly recent assumption in runtest.rs that all MIR dump
files end in .mir. (It was appending .mir to the graphviz .dot and
spanview .html file names when generating blessed output files. That
also left outdated files in the baseline alongside the files with the
incorrect names, which I've now removed.)
Updated spanview HTML title elements to match their content, replacing a
hardcoded and incorrect name that was left in accidentally when
originally submitted.
Commit #4:
added more test examples
also improved Makefiles with support for non-zero exit status and to
force validation of tests unless a specific test overrides it with a
specific comment.
Commit #5:
Fixed rare issues after testing on real-world crate
Commit #6:
Addressed PR feedback, and removed temporary -Zexperimental-coverage
-Zinstrument-coverage once again supports the latest capabilities of
LLVM instrprof coverage instrumentation.
Also fixed a bug in spanview.
Commit #7:
Fix closure handling, add tests for closures and inner items
And cleaned up other tests for consistency, and to make it more clear
where spans start/end by breaking up lines.
Commit #8:
renamed "typical" test results "expected"
Now that the `llvm-cov show` tests are improved to normally expect
matching actuals, and to allow individual tests to override that
expectation.
Commit #9:
test coverage of inline generic struct function
Commit #10:
Addressed review feedback
* Removed unnecessary Unreachable filter.
* Replaced a match wildcard with remining variants.
* Added more comments to help clarify the role of successors() in the
CFG traversal
Commit #11:
refactoring based on feedback
* refactored `fn coverage_spans()`.
* changed the way I expand an empty coverage span to improve performance
* fixed a typo that I had accidently left in, in visit.rs
Commit #12:
Optimized use of SourceMap and SourceFile
Commit #13:
Fixed a regression, and synched with upstream
Some generated test file names changed due to some new change upstream.
Commit #14:
Stripping out crate disambiguators from demangled names
These can vary depending on the test platform.
Commit #15:
Ignore llvm-cov show diff on test with generics, expand IO error message
Tests with generics produce llvm-cov show results with demangled names
that can include an unstable "crate disambiguator" (hex value). The
value changes when run in the Rust CI Windows environment. I added a sed
filter to strip them out (in a prior commit), but sed also appears to
fail in the same environment. Until I can figure out a workaround, I'm
just going to ignore this specific test result. I added a FIXME to
follow up later, but it's not that critical.
I also saw an error with Windows GNU, but the IO error did not
specify a path for the directory or file that triggered the error. I
updated the error messages to provide more info for next, time but also
noticed some other tests with similar steps did not fail. Looks
spurious.
Commit #16:
Modify rust-demangler to strip disambiguators by default
Commit #17:
Remove std::process::exit from coverage tests
Due to Issue #77553, programs that call std::process::exit() do not
generate coverage results on Windows MSVC.
Commit #18:
fix: test file paths exceeding Windows max path len
Replace `(Body, DefId)` with `Body` where possible
Follow-up to #77430.
I `grep`-ed for parameter lists in which a `Body` appeared within a few lines of a `DefId`, so it's possible that I missed some cases, but this should be pretty complete. Most of these changes were mechanical, but there's a few places where I started calling things "caller" and "callee" when multiple `DefId`s were in-scope at once. Also, we should probably have a helper function on `Body` that returns a `LocalDefId`. I can do that in this PR or in a follow-up.
Improve build-manifest to work with the improved promote-release
This PR makes some changes to build-manifest to have it work better with the other improvements I'm making to [promote-release](https://github.com/rust-lang/promote-release).
A new way to invoke the tool was added: `./x.py run src/tools/build-manifest`. The new invocation disables the generation of `.sha256` files and the generation of GPG signatures, as those steps are not tied to the Rust version we're building the manifest of: handling them in `promote-release` will improve the maintenability of our release process. Invocations through the old command (`./x.py dist hash-and-sign`) are referred inside the source code as "legacy". The new invocation also enables internal parallelism, disabled on legacy to avoid overloading our old server.
Improvements were also made on how the checksums included in the manifest are generated:
* The manifest is first generated with placeholder checksums, and then a function walks through the manifes and calculates only the needed hashes. Before this PR, all the hashes were calculated beforehand, including the hashes of unused files.
* Calculating the hashes is now done in parallel with rayon, to better utilize all the available disk bandwidth.
* The `sha2` crate is now used instead of the `sha256sum` CLI tool: this avoids the overhead of calling another process, but more importantly enables hardware acceleration whenever available (the `sha256sum` CLI tool doesn't support it at all).
r? @Mark-Simulacrum
This PR is best reviewed commit-by-commit.
Use `tracing` spans to trace the entire MIR interp stack
r? @RalfJung
While being very verbose, this allows really good tracking of what's going on. While I considered schemes like the previous indenter that we had (which we could get by using the `tracing-tree` crate), this will break down horribly with things like multithreaded rustc. Instead, we can now use `RUSTC_LOG` to restrict the things being traced. You could specify a filter in a way that only shows the logging of a specific frame.
If we lower the span's level to `debug`, then in `info` level logging we'd not see the frames, but in `debug` level we would see them. The filtering rules in `tracing` are super powerful, but I'm not sure if we can specify a filter so we do see `debug` level events, but *not* the `frame` spans. The documentation at https://docs.rs/tracing-subscriber/0.2.10/tracing_subscriber/struct.EnvFilter.html makes me think that we can only turn on things, not turn off things at a more precise level.
cc @hawkw
Write manifest for MAJOR.MINOR channel to enable rustup convenience
This connects to https://github.com/rust-lang/rustup/issues/794.
It's hard to remember if there have been patch releases for old versions
when you'd like to install the latest in a MAJOR.MINOR series.
When we're doing a stable release, we write duplicate manifests to
`stable`. With this change, only when we're doing a stable release, also
write duplicate manifests to `MAJOR.MINOR` to eventually enable rustup
(and any other tooling that builds Rust release URLs) to request, say,
`1.45` and get `1.45.2` (assuming `1.45.2` is the latest available
`1.45` and assuming that we never publish patch releases out of order).
I tested the best I could; it's a bit hard to get everything set up right
to be able to run the build-manifest tool. But I was able to run it with
a release of "1.45.2" and in addition to the files like `channel-rust-1.45.2.toml`
and `channel-rust-stable.toml` (and other manifests) that I got before this
change, I now get `channel-rust-1.45.toml`.
I believe this change to be safe to deploy as it does not change or remove
anything about manifests, just adds more. The actions in rust-central-station
that interact with manifests appear to use wildcards in such a way that it will
pick up these files without any problems.
There will need to be changes to `rustup` before `rustup install 1.45` will work,
but we can wait for a stable release and stable patch releases to happen with this
change before making the `rustup` changes, so that we're not committing to anything
before we know it works.
Rollup of 11 pull requests
Successful merges:
- #76851 (Fix 'FIXME' about using NonZeroU32 instead of u32.)
- #76979 (Improve std::sys::windows::compat)
- #77111 (Stabilize slice_ptr_range.)
- #77147 (Split sys_common::Mutex in StaticMutex and MovableMutex.)
- #77312 (Remove outdated line from `publish_toolstate` hook)
- #77362 (Fix is_absolute on WASI)
- #77375 (rustc_metadata: Do not forget to encode inherent impls for foreign types)
- #77385 (Improve the example for ptr::copy)
- #77389 (Fix some clippy lints)
- #77399 (BTreeMap: use Unique::from to avoid a cast where type information exists)
- #77429 (Link `new` method in `DefautHasher`s doc)
Failed merges:
r? `@ghost`
Remove outdated line from `publish_toolstate` hook
We no longer add `I-nominated` to toolstate failure issues since T-compiler changed its meeting preparation workflow.
This commit improves the way build-manifest calculates the checksums
included in the manifest, speeding it up:
* Instead of calculating all the hashes beforehand and then using the
ones we need, the manifest is first generated with placeholder hashes,
and then a function walks through the manifest and calculates only the
needed checksums.
* Calculating the checksums is now done in parallel with rayon, to
better utilize all the available disk bandwidth.
* Calculating the checksums now uses the sha2 crate instead of the
sha256sum CLI tool: this avoids the overhead of calling another
process, but more importantly uses hardware acceleration whenever
available (the CLI tool doesn't support it at all).
This connects to https://github.com/rust-lang/rustup/issues/794.
It's hard to remember if there have been patch releases for old versions
when you'd like to install the latest in a MAJOR.MINOR series.
When we're doing a stable release, we write duplicate manifests to
`stable`. With this change, only when we're doing a stable release, also
write duplicate manifests to `MAJOR.MINOR` to eventually enable rustup
(and any other tooling that builds Rust release URLs) to request, say,
`1.45` and get `1.45.2` (assuming `1.45.2` is the latest available
`1.45` and assuming that we never publish patch releases out of order).
Defer Apple SDKROOT detection to link time.
This defers the detection of the SDKROOT for Apple iOS/tvOS targets to link time, instead of when the `Target` is defined. This allows commands that don't need to link to work (like `rustdoc` or `rustc --print=target-list`). This also makes `--print=target-list` a bit faster.
This also removes the note in the platform support documentation about these targets being missing. When I wrote it, I misunderstood how the SDKROOT stuff worked.
Notes:
* This means that JSON spec targets can't explicitly override these flags. I think that is probably fine, as I believe the value is generally required, and can be set with the SDKROOT environment variable.
* This changes `x86_64-apple-tvos` to use `appletvsimulator`. I think the original code was wrong (it was using `iphonesimulator`). Also, `x86_64-apple-tvos` seems broken in general, and I cannot build it locally. The `data_layout` does not appear to be correct (it is a copy of the arm64 layout instead of the x86_64 layout). I have not tried building Apple's LLVM to see if that helps, but I suspect it is just wrong (I'm uncertain since I don't know how the tvOS simulator works with its bitcode-only requirements).
* I'm tempted to remove the use of `Result` for built-in target definitions, since I don't think they should be fallible. This was added in https://github.com/rust-lang/rust/pull/34980, but that only relates to JSON definitions. I think the built-in targets shouldn't fail. I can do this now, or not.
Fixes#36156Fixes#76584
Ensure that all LLVM components requested by tests are available on CI
Addresses https://github.com/rust-lang/rust/pull/75064#issuecomment-667722652
I used an environment variable because passing a command line option all the way from CI to compiletest would be just too much hassle for this task.
I added a new variable, but any of the already existing ones defined by CI could be used instead.
r? @Mark-Simulacrum
Refactor versions detection in build-manifest
This PR refactors how `build-manifest` handles versions, making the following changes:
* `build-manifest` now detects the "package releases" on its own, without relying on rustbuild providing them through CLI arguments. This drastically simplifies calling the tool outside of `x.py`, and will allow to ship the prebuilt tool in a tarball in the future, with the goal of stopping to invoke `x.py` during `promote-release`.
* The `tar` command is not used to extract the version and the git hash from tarballs anymore. The `flate2` and `tar` crates are used instead. This makes detecting those pieces of data way faster, as the archive is decompressed just once and we stop parsing the archive once all the information is retrieved.
* The code to extract the version and the git hash now stores all the collected data dynamically, without requiring to add new fields to the `Builder` struct every time.
I tested the changes locally and it should behave the same as before.
r? `@Mark-Simulacrum`
Add option to pass a custom codegen backend from a driver
This allows the driver to pass information to the codegen backend. For example the headcrab debugger may in the future want to use cg_clif to JIT code to be injected in the debuggee. This would PR make it possible to tell cg_clif which symbol can be found at which address and to tell it to inject the JITed code into the right process.
This PR may also help with https://github.com/rust-lang/miri/pull/1540 by allowing miri to provide a codegen backend that only emits metadata and doesn't perform any codegen.
cc @nbaksalyar (headcrab)
cc @RalfJung (miri)
might_permit_raw_init: also check aggregate fields
This is the next step for https://github.com/rust-lang/rust/issues/66151: when doing `mem::zeroed`/`mem::uninitialized`, also recursively check fields of aggregates (except for arrays) for whether they permit zero/uninit initialization.
Move helper function for `missing_const_for_fn` out of rustc to clippy
cc @rust-lang/clippy @ecstatic-morse #76618
r? @Manishearth
I also removed all support for suggesting a function could be `const fn` when that would require feature gates to actually work.
This means we'll now have to maintain this ourselves in clippy, but that's how most lints work anyway, so...
Update cargo
7 commits in 8777a6b1e8834899f51b7e09cc9b8d85b2417110..05c611ae3c4255b7a2bcf4fcfa65b20286a07839
2020-09-15 19:11:03 +0000 to 2020-09-23 23:10:38 +0000
- --workspace flag for locate-project to find the workspace root (rust-lang/cargo#8712)
- Remove some badges documentation. (rust-lang/cargo#8727)
- Add plain message format for locate-project (rust-lang/cargo#8707)
- Add a term option to configure the progress bar (rust-lang/cargo#8165)
- Replace d_as_f64 with as_secs_f64 (rust-lang/cargo#8721)
- Add cross check to filters_target test. (rust-lang/cargo#8713)
- Add test for whitespace behavior in env flags. (rust-lang/cargo#8706)
