CI: dynamic jobs
This PR modifies our CI workflows to be dynamic. This means that when a GitHub event is generated, we will run a Python script (`calculate-job-matrix.py`), which decides which CI jobs should be generated. These jobs are defined in `src/ci/github-actions/jobs.yml`).
This should provide a few benefits:
- Once the migration to dynamic jobs is complete, we shouldn't need `expand-yaml-anchors` anymore.
- The job table on PRs (and also the left job column on auto/try builds) should be much cleaner and contain only the jobs that are actually relevant/executed.
- It should be much easier to support dynamic try builds, i.e. to run an arbitrary CI job on a try build.
See [this Zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/job.20matrix.20re-ordered.20PR.20list) for more context.
r? `@ghost`
Rollup of 3 pull requests
Successful merges:
- #124003 (Dellvmize some intrinsics (use `u32` instead of `Self` in some integer intrinsics))
- #124169 (Don't fatal when calling `expect_one_of` when recovering arg in `parse_seq`)
- #124286 (Subtree sync for rustc_codegen_cranelift)
r? `@ghost`
`@rustbot` modify labels: rollup
Subtree sync for rustc_codegen_cranelift
This fixes a crash when compiling the standard library. In addition the Cranelift update fixes all the 128bit int abi incompatibility between cg_clif and cg_llvm.
r? ``@ghost``
``@rustbot`` label +A-codegen +A-cranelift +T-compiler
Dellvmize some intrinsics (use `u32` instead of `Self` in some integer intrinsics)
This implements https://github.com/rust-lang/compiler-team/issues/693 minus what was implemented in #123226.
Note: I decided to _not_ change `shl`/... builder methods, as it just doesn't seem worth it.
r? ``@scottmcm``
Disallow ambiguous attributes on expressions
This implements the suggestion in [#15701](https://github.com/rust-lang/rust/issues/15701#issuecomment-2033124217) to disallow ambiguous outer attributes on expressions. This should resolve one of the concerns blocking the stabilization of `stmt_expr_attributes`.
Rollup of 7 pull requests
Successful merges:
- #123680 (Deny gen keyword in `edition_2024_compat` lints)
- #124057 (Fix ICE when ADT tail has type error)
- #124168 (Use `DefiningOpaqueTypes::Yes` in rustdoc, where the `InferCtxt` is guaranteed to have no opaque types it can define)
- #124197 (Move duplicated code in functions in `tests/rustdoc-gui/notable-trait.goml`)
- #124200 (Improve handling of expr->field errors)
- #124220 (Miri: detect wrong vtables in wide pointers)
- #124266 (remove an unused type from the reentrant lock tests)
r? `@ghost`
`@rustbot` modify labels: rollup
Deny gen keyword in `edition_2024_compat` lints
Splits the `keyword_idents` lint into two -- `keyword_idents_2018` and `keyword_idents_2024` -- since each corresponds to a future-compat warning in a different edition. Group these together into a new `keyword_idents` lint group, and add the latter to the `rust_2024_compatibility` so that `gen` is ready for the 2024 edition.
cc `@traviscross` `@ehuss`
Add simple async drop glue generation
This is a prototype of the async drop glue generation for some simple types. Async drop glue is intended to behave very similar to the regular drop glue except for being asynchronous. Currently it does not execute synchronous drops but only calls user implementations of `AsyncDrop::async_drop` associative function and awaits the returned future. It is not complete as it only recurses into arrays, slices, tuples, and structs and does not have same sensible restrictions as the old `Drop` trait implementation like having the same bounds as the type definition, while code assumes their existence (requires a future work).
This current design uses a workaround as it does not create any custom async destructor state machine types for ADTs, but instead uses types defined in the std library called future combinators (deferred_async_drop, chain, ready_unit).
Also I recommend reading my [explainer](https://zetanumbers.github.io/book/async-drop-design.html).
This is a part of the [MCP: Low level components for async drop](https://github.com/rust-lang/compiler-team/issues/727) work.
Feature completeness:
- [x] `AsyncDrop` trait
- [ ] `async_drop_in_place_raw`/async drop glue generation support for
- [x] Trivially destructible types (integers, bools, floats, string slices, pointers, references, etc.)
- [x] Arrays and slices (array pointer is unsized into slice pointer)
- [x] ADTs (enums, structs, unions)
- [x] tuple-like types (tuples, closures)
- [ ] Dynamic types (`dyn Trait`, see explainer's [proposed design](https://github.com/zetanumbers/posts/blob/main/async-drop-design.md#async-drop-glue-for-dyn-trait))
- [ ] coroutines (https://github.com/rust-lang/rust/pull/123948)
- [x] Async drop glue includes sync drop glue code
- [x] Cleanup branch generation for `async_drop_in_place_raw`
- [ ] Union rejects non-trivially async destructible fields
- [ ] `AsyncDrop` implementation requires same bounds as type definition
- [ ] Skip trivially destructible fields (optimization)
- [ ] New [`TyKind::AdtAsyncDestructor`](https://github.com/zetanumbers/posts/blob/main/async-drop-design.md#adt-async-destructor-types) and get rid of combinators
- [ ] [Synchronously undroppable types](https://github.com/zetanumbers/posts/blob/main/async-drop-design.md#exclusively-async-drop)
- [ ] Automatic async drop at the end of the scope in async context
Update books
## rust-lang/book
41 commits in 3131aa4642c627a24f523c82566b94a7d920f68c..d207d894cc5e1d496ab99beeacd1a420e5d4d238
2024-04-19 19:48:19 UTC to 2024-04-08 19:34:07 UTC
- Ch. 2: intentionally use `{}` at first (rust-lang/book#3898)
- Inline fmt lints for `redirects/` (rust-lang/book#3897)
- ch12-03: `expect` is not used multitude times by this chapter (rust-lang/book#3287)
- Inline format args to `tools/src` (rust-lang/book#3896)
- Improve awkward phrasing around the kinds of closures (rust-lang/book#3290)
- Ch. 14: clarify workspace dependency reuse (rust-lang/book#3890)
- Edit on line no. 153. Replaced fn add_to_waitlist with fn eat_at_restaurant. (rust-lang/book#3281)
- Ch. 2: less ambiguity about numbers (rust-lang/book#3895)
- Clarified first slices paragraph (rust-lang/book#3363)
- Inline all format arguments (rust-lang/book#3385)
- Use v1.58 captured ident formatting in examples (rust-lang/book#3048)
- Revert "ch18-03: Guarded match arm exhaustivness clarification" (rust-lang/book#3894)
- Ch. 18: further clarify about irrefutable patterns (rust-lang/book#3893)
- ch18-02: Minor corrections (rust-lang/book#3381)
- Make style of listings 9-7 and 9-8 consistent with 9-6 (rust-lang/book#3217)
- ch18-03: Guarded match arm exhaustivness clarification (rust-lang/book#3397)
- ch11-03: Unnecessary import removal (rust-lang/book#3284)
- Update ch20-01-single-threaded.md (rust-lang/book#3394)
- In Appendix D, adds hyperlink to Appendix E (rust-lang/book#3418)
- Chapter 13-03 fixed Iterator 'type' to 'trait' (rust-lang/book#3433)
- Fix the expected substring in ch11-01 (rust-lang/book#3534)
- Ch. 17: clarify 'in order to compile' (rust-lang/book#3892)
- Revert "Chapter 15-05: Fix incorrect interpretation of compiler error" (rust-lang/book#3891)
- Chapter 15-05: Fix incorrect interpretation of compiler error (rust-lang/book#3501)
- Remove direction to authenticate with `cargo login {API token}` (rust-lang/book#3506)
- Update syn dependencies in Chapter 19 to 2.0 (rust-lang/book#3888)
- Chapter 3: Clarify `return` vs. `break`. (rust-lang/book#3889)
- Fix "`Display` type" to "`Display` trait" in ch19-03 (rust-lang/book#3835)
- Update listings in ch 11-01 to reflect current output from cargo new (rust-lang/book#3533)
- Fix "`test` module" to "`tests` module" in ch11-03 (rust-lang/book#3579)
- Ch. 2: fix wording about `cargo update` (rust-lang/book#3882)
- Remove hebrew niqqud in 8-14 String::from example (rust-lang/book#3648)
- Ch. 16: fix a grammar issue (rust-lang/book#3881)
- Update MSVC instructions (rust-lang/book#3616)
- Replace RLS with rust-analyzer in ch00 (rust-lang/book#3546)
- Fix em dashes (rust-lang/book#3570)
- Revert #3711 (rust-lang/book#3880)
- Procedure Macro: Update dependencies (rust-lang/book#3631)
- Parity with rustup's help (rust-lang/book#3642)
- Fix confusing sentence (rust-lang/book#3877)
- Ch. 7: add a note about the need for `pub` in one more place (rust-lang/book#3879)
## rust-lang/edition-guide
2 commits in eb3eb80e106d03250c1fb7c5666b1c8c59672862..0c68e90acaae5a611f8f5098a3c2980de9845ab2
2024-04-16 21:23:55 UTC to 2024-04-15 16:05:32 UTC
- Document RustcEncodable/RustcDecodable (rust-lang/edition-guide#298)
- Flesh out detail behind disallowing refs to static mut (rust-lang/edition-guide#299)
## rust-lang/reference
5 commits in 55694913b1301cc809f9bf4a1ad1b3d6920efbd9..5854fcc286557ad3ab34d325073d11d8118096b6
2024-04-21 13:47:07 UTC to 2024-04-15 16:12:40 UTC
- Stabilize Wasm target features that are in phase 4 and 5 (rust-lang/reference#1420)
- Expand and clarify primitive alignment (rust-lang/reference#1493)
- Update clone reference to include closures (rust-lang/reference#1492)
- Document how `non_exhaustive` interacts with tuple and unit-like structs. (rust-lang/reference#1491)
- Fix link to RISC-V Zkt spec; it was pointing to Zkr (rust-lang/reference#1490)
## rust-lang/rustc-dev-guide
4 commits in b77a34bd46399687b4ce6a17198e9f316c988794..07425fed36b00e60341c5e29e28d37d40cbd4451
2024-04-13 15:16:44 UTC to 2024-04-08 19:00:54 UTC
- Add a note about checks with multiple revisions (rust-lang/rustc-dev-guide#1963)
- compiletest: clarify that headers must be one header per line (rust-lang/rustc-dev-guide#1962)
- chore: fix some typos in conments (rust-lang/rustc-dev-guide#1961)
- compiletest: specify which special env var and which particular CI job (rust-lang/rustc-dev-guide#1960)
Ignore `-C strip` on MSVC
tl;dr - Define `-Cstrip` to only ever affect the binary; no other build artifacts.
This is necessary to improve cross-platform behavior consistency: if someone wanted debug information to be contained only in separate files on all platforms, they would set `-Cstrip=symbols` and `-Csplit-debuginfo=packed`, but this would result in no PDB files on MSVC.
Resolves#114215
Rollup of 4 pull requests
Successful merges:
- #124069 (enable clippy for bootstrap on CI PRs (in `mingw-check` image))
- #124089 (Fix watchOS and visionOS for pread64 and pwrite64 calls)
- #124184 (Suggest using `unsigned_abs` in `abs` documentation)
- #124198 (Flip spans for precise capturing syntax not capturing a ty/const param, and for implicit captures of lifetime params)
r? `@ghost`
`@rustbot` modify labels: rollup
CI: add script for installing NodeJS and update it to v20
I centralized the installation on a single place to make it simple to update the NodeJS version across the board.
Fixes: https://github.com/rust-lang/rust/issues/123965
r? `@Mark-Simulacrum`
mir-opt tests: rename unit-test -> test-mir-pass
"unit-test" is extremely non-descriptive, no idea how one is supposed to read that and know that this specifies the MIR pass being tested.
Add llvm-bitcode-linker to build manifest
When creating #123423 I didn't realize I also had to add the new component to the build-manifest. This PR finishes the work of adding it, by also adding it to the build manifest.
r? ``@Mark-Simulacrum``
lint-docs: Add redirects for renamed lints.
This updates the lint docs to include a redirect for renamed lints to the new name. This helps ensure that links to the old name will still be valid.
Note that this currently uses a hard-coded list. As mentioned in the comment, a future enhancement may gather this information in a better way.
Unblocks #123680
Add a lower bound check to `unicode-table-generator` output
This adds a dedicated check for the lower bound
(if it is outside of ASCII range) to the output of the `unicode-table-generator` tool.
This generalized the ASCII-only fast-path, but only for the `Grapheme_Extend` property for now, as that is the only one with a lower bound outside of ASCII.
Match hyphen in multi-revision comment matchers
Currently, the matcher `//[rev-foo,rev-bar]~` does not get selected by the regex. Change the matcher to include `-`.
Implement Modified Condition/Decision Coverage
This is an implementation based on llvm backend support (>= 18) by `@evodius96` and branch coverage support by `@Zalathar.`
### Major changes:
* Add -Zcoverage-options=mcdc as switch. Now coverage options accept either `no-branch`, `branch`, or `mcdc`. `mcdc` also enables `branch` because it is essential to work.
* Add coverage mapping for MCDCBranch and MCDCDecision. Note that MCDCParameter evolves from llvm 18 to llvm 19. The mapping in rust side mainly references to 19 and is casted to 18 types in llvm wrapper.
* Add wrapper for mcdc instrinc functions from llvm. And inject associated statements to mir.
* Add BcbMappingKind::Decision, I'm not sure is it proper but can't find a better way temporarily.
* Let coverage-dump support parsing MCDCBranch and MCDCDecision from llvm ir.
* Add simple tests to check whether mcdc works.
* Same as clang, currently rustc does not generate instrument for decision with more than 6 condtions or only 1 condition due to considerations of resource.
### Implementation Details
1. To get information about conditions and decisions, `MCDCState` in `BranchInfoBuilder` is used during hir lowering to mir. For expressions with logical op we call `Builder::visit_coverage_branch_operation` to record its sub conditions, generate condition ids for them and save their spans (to construct the span of whole decision). This process mainly references to the implementation in clang and is described in comments over `MCDCState::record_conditions`. Also true marks and false marks introduced by branch coverage are used to detect where the decision evaluation ends: the next id of the condition == 0.
2. Once the `MCDCState::decision_stack` popped all recorded conditions, we can ensure that the decision is checked over and push it into `decision_spans`. We do not manually insert decision span to avoid complexity from then_else_break in nested if scopes.
3. When constructing CoverageSpans, add condition info to BcbMappingKind::Branch and decision info to BcbMappingKind::Decision. If the branch mapping has non-zero condition id it will be transformed to MCDCBranch mapping and insert `CondBitmapUpdate` statements to its evaluated blocks. While decision bcb mapping will insert `TestVectorBitmapUpdate` in all its end blocks.
### Usage
```bash
echo "[build]\nprofiler=true" >> config.toml
./x build --stage 1
./x test tests/coverage/mcdc_if.rs
```
to build the compiler and run tests.
```shell
export PATH=path/to/llvm-build:$PATH
rustup toolchain link mcdc build/host/stage1
cargo +mcdc rustc --bin foo -- -Cinstrument-coverage -Zcoverage-options=mcdc
cd target/debug
LLVM_PROFILE_FILE="foo.profraw" ./foo
llvm-profdata merge -sparse foo.profraw -o foo.profdata
llvm-cov show ./foo -instr-profile=foo.profdata --show-mcdc
```
to check "foo" code.
### Problems to solve
For now decision mapping will insert statements to its all end blocks, which may be optimized by inserting a final block of the decision. To do this we must also trace the evaluated value at each end of the decision and join them separately.
This implementation is not heavily tested so there should be some unrevealed issues. We are going to check our rust products in the next. Please let me know if you had any suggestions or comments.
