CI: add a script for dynamically computing CI job matrix
It would be great if was easier to run specific CI workflows locally, and also to allow us to spawn a specific CI workflow by bors, to enable running arbitrary try builds. See discussion [here](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/CI.20workflows.20refactoring).
This PR is a first step in that direction.
- Moves the definition of CI runners and (for now) PR jobs into a separate `jobs.yml` file.
- Adds a simple Python script that reads the file, decides which jobs should be active for the current CI workflow, and prints them as JSON to their output.
- The PR job then reads this output and generates its job matrix based on it.
By moving the job definitions from `ci.yml` into a separate file, we can handle it programmatically, which should make it easier to both do local execution of CI jobs and also to do arbitrary try builds.
Make `split_simd_to_128bit_chunks` take only one operand
It will allow more flexible uses in the future. This makes `split_simd_to_128bit_chunks` simpler, moving some of the complexity to its callers.
Rollup of 12 pull requests
Successful merges:
- #123423 (Distribute LLVM bitcode linker as a preview component)
- #123548 (libtest: also measure time in Miri)
- #123666 (Fix some typos in doc)
- #123864 (Remove a HACK by instead inferring opaque types during expected/formal type checking)
- #123896 (Migrate some diagnostics in `rustc_resolve` to session diagnostic)
- #123919 (builtin-derive: tag → discriminant)
- #123922 (Remove magic constants when using `base_n`.)
- #123931 (Don't leak unnameable types in `-> _` recover)
- #123933 (move the LargeAssignments lint logic into its own file)
- #123934 (`rustc_data_structures::graph` mini refactor)
- #123941 (Fix UB in LLVM FFI when passing zero or >1 bundle)
- #123957 (disable create_dir_all_bare test on all(miri, windows))
r? `@ghost`
`@rustbot` modify labels: rollup
libtest: also measure time in Miri
A long time ago we disabled timekeeping of the default test harness in Miri, as otherwise it would fail to run without `-Zmiri-disable-isolation`. However, since then Miri gained a "fake clock" that lets it present some deterministic notion of time when isolation is enabled.
So we could now let libtest do timekeeping again when running in Miri. That's nice as it can help detect tests that run too long. However it can also be confusing as the results with isolation can be quite different than the real time.
``@rust-lang/miri`` what do you think?
Handle Miri sysroot entirely outside the Miri driver
(Extracted from https://github.com/rust-lang/miri/pull/3409)
This entirely moves the responsibility of setting miri-sysroot to whatever *invokes* the Miri driver. cargo-miri knows whether it is inside rustdoc or not and can adjust accordingly. I previously avoided doing that because there are a bunch of places that are invoking the driver (cargo-miri, the ui test suite, `./miri run`, `./x.py run miri`) and they all need to be adjusted now. But it is also somewhat less fragile as we usually have more information there -- and we can just decide that `./miri run file.rs --sysroot path` is not supported. The advantage of this is that the driver is reasonably clean and doesn't need magic environment variables like MIRI_SYSROOT, and we don't have to fight rustc_driver to use a different default sysroot. Everything is done in cargo-miri (and the other much simpler driver wrappers) where it can hopefully be debugged much better.
Miri on Windows: run .CRT$XLB linker section on thread-end
Hopefully fixes https://github.com/rust-lang/rust/issues/123583
First commit is originally by `@bjorn3`
r? `@oli-obk`
Cc `@ChrisDenton`
compiletest ice tracking
see https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/where.20to.20mass-add.20known.20ices.20.2F.20merging.20glacier.20into.20rust/near/429082963
This will allow us to sunset most of https://github.com/rust-lang/glacier
The rustc ices will be tracked directly inside the rust testsuite
There are a couple of .sh tests remaining that I have not ported over yet.
This adds `tests/crashes`, a file inside this directory MUST ice, otherwise it is considered test-fail.
This will be used to track ICEs from glacier and the bugtracker.
When someones pr accidentally fixes one of these ICEs, they can move the test from `crashes` into `ui` for example.
I also added a new tidy lint that warns when a test inside `tests/crashes` does not have a `//@ known-bug: ` line
the env var `COMPILETEST_VERBOSE_CRASHES` can be set to get exit code, stderr and stdout of a crash-test to aid debugging/adding tests.
ci: test cargo on `aarch64-gnu`
Since `aarch64-unknown-linux-gnu` is a tier-1 target, we should also test cargo on it, especially since cargo's own CI doesn't cover this yet. This might have helped us discover #123733 sooner, which is not a cargo problem but was uncovered by a new cargo test (which we'll have to skip for now). Everything else passes in my local run, so at least we'll have a guard against future regressions.
skip Codegen{GCC,Cranelift} when using CI rustc
CI rustc uses the default codegen backend, therefore we can't run `CodegenGCC` and `CodegenCranelift` tests when using it.
cc `@bjorn3` (to make sure I am not doing anything wrong)
Fixes#123331
previously we would explicitly look for exit code 101 and call it a crash,
however in case of stack overflows for example, exit code could differ due to the
process being killed by a signal which is not easy to detect on none-unix.
So now we reject everything that exits with 0 (no error) or 1 (compiler failed to compile code)
and "accept" everyhing else as an internal compiler error.
compiletest: Update rustfix to 0.8.1
This updates the version of rustfix used in compiletest to be closer to what cargo is using. This is to help ensure `cargo fix` and compiletest are aligned. There are some unpublished changes to `rustfix`, which will update in a future PR when those are published.
Will plan to update ui_test in the near future to avoid the duplicate.
reduce tidy overheads in run-make checks
This change makes tidy to handle run-make checks with a single iteration, avoiding the need for multiple iterations and copying.
run-make-support: tidy up support library
- Make `handle_failed_output` take `&Command` instead of having the caller keep doing `format!("{:#?}", s)`.
- Introduce a helper macro for implementing common command wrappers, such as `arg`, `args`, `run`, `run_fail`.
- Use the helper macro on existing command wrappers and remove manual copy-pasta'd implementations.
