Commit graph

1943 commits

Author SHA1 Message Date
Matthias Krüger
4abf179b14
Rollup merge of #125011 - diondokter:opt-for-size, r=Amanieu,kobzol
Add opt-for-size core lib feature flag

Adds a feature flag to the core library that enables the possibility to have smaller implementations for certain algorithms.

So far, the core lib has traded performance for binary size. This is likely what most people want since they have big simd-capable machines. However, people on small machines, like embedded devices, don't enjoy the potential speedup of the bigger algorithms, but do have to pay for them. These microcontrollers often only have 16-1024kB of flash memory.

This PR is the result of some talks with project members like `@Amanieu` at RustNL.
There are some open questions of how this is eventually stabilized, but it's a similar question as with the existing `panic_immediate_abort` feature.

Speaking as someone from the embedded side, we'd rather have this unstable for a while as opposed to not having it at all. In the meantime we can try to use it and also add additional PRs to the core lib that uses the feature flag in areas where we find benefit.

Open questions from my side:
- Is this a good feature name?
  - `panic_immediate_abort` is fairly verbose, so I went with something equally verbose
  - It's easy to refactor later
- I've added the feature to `std` and `alloc` as well as they might benefit too. Do we agree?
  - I expect these to get less usage out of the flag since most size-constraint projects don't use these libraries often.
2024-05-21 12:47:04 +02:00
Bernardo Meurer Costa
d9a7868ceb refactor(ci/docker/dist-x86_64-linux): remove rustc-perf downloading
It is now available as a submodule in src/tools/rustc-perf, and is
initialized when building opt-dist
2024-05-20 15:01:18 +00:00
bors
44d679b902 Auto merge of #124985 - dpaoliello:rebalance, r=Mark-Simulacrum
Reduce builder size of jobs that take less than an hour

The current longest build time is ~2hr for the `dist-x86_64-linux-alt` build. This is already on a 16-core builder, so we can't make it any faster (by throwing more hardware at it).

Given that overall build times will be at least 2hrs, we can reduce build costs by reducing the builder size for any job that takes less than 1hr since it will still complete before `dist-x86_64-linux-alt` does.

Note that scaling isn't linear, halving the core count increases end-to-end build times by about 25-50%. In [this sample build](https://github.com/rust-lang/rust/actions/runs/9037235792/usage?pr=124985) `arm-android` went from ~52m to 1h 5m and `dist-arm-linux` went from ~55m to 1h 17m (then failed due to missing metrics).

Current job builder sizes and times and proposed new sizes:

| Job | Size | Proposed | Run 1 | Run 2 | Run 3 | Run 4 |
|-----|------|----------|-------|-------|-------|-------|
| aarch64-gnu | - | | 1h 9m 1s | 1h 8m 47s | 1h 8m 45s | 1h 9m 6s |
| arm-android | 8c | 4c | 52m 32s | 52m 38s | 51m 30s | 53m 13s |
| armhf-gnu | 8c | 4c | 37m 30s | 37m 40s | 38m 41s | 37m 56s |
| dist-aarch64-linux | 8c | 4c | 57m 11s | 56m 48s | 55m 53s | 56m 19s |
| dist-android | 8c | 4c | 24m 37s | 25m 13s | 25m 15s | 24m 17s |
| dist-arm-linux | 16c | 8c | 53m 34s | 55m 11s | 56m 1s | 54m 29s |
| dist-armhf-linux | 8c | 4c | 42m 1s | 43m 32s | 43m 27s | 41m 55s |
| dist-armv7-linux | 8c | 4c | 44m 51s | 44m 35s | 43m 34s | 46m 2s |
| dist-i586-gnu-i586-i686-musl | 8c | 4c | 37m 59s | 37m 56s | 38m 4s | 38m 24s |
| dist-i686-linux | 8c | 4c | 52m 20s | 51m 3s | 52m 53s | 50m 38s |
| dist-loongarch64-linux | 8c | 4c | 40m 39s | 40m 20s | 41m 6s | 40m 44s |
| dist-ohos | 8c | 4c | 25m 5s | 24m 34s | 25m 18s | 23m 40s |
| dist-powerpc-linux | 8c | 4c | 42m 31s | 43m 53s | 42m 35s | 41m 56s |
| dist-powerpc64-linux | 8c | 4c | 42m 52s | 44m 36s | 45m 32s | 43m 51s |
| dist-powerpc64le-linux | 8c | 4c | 43m 41s | 44m 11s | 43m 2s | 44m 21s |
| dist-riscv64-linux | 8c | 4c | 41m 25s | 42m 41s | 41m 52s | 43m 47s |
| dist-s390x-linux | 8c | 4c | 46m 48s | 47m 18s | 47m 27s | 46m 49s |
| dist-various-1 | 8c | 4c | 42m 14s | 43m 20s | 43m 20s | 41m 41s |
| dist-various-2 | 8c | 4c | 36m 18s | 38m 15s | 37m 41s | 39m 28s |
| dist-x86_64-freebsd | 8c | 4c | 39m 21s | 39m 40s | 40m 1s | 40m 2s |
| dist-x86_64-illumos | 8c | 4c | 45m 35s | 46m 43s | 46m 2s | 46m 4s |
| dist-x86_64-linux | 16c | | 1h 53m 10s | 1h 51m 15s | 1h 52m 18s | 1h 52m 26s |
| dist-x86_64-linux-alt | 16c | | 2h 3m 33s | 2h 3m 31s | 2h 4m 12s | 2h 2m 21s |
| dist-x86_64-musl | 8c | | 1h 5m 42s | 1h 6m 13s | 1h 7m 49s | 1h 6m 6s |
| dist-x86_64-netbsd | 8c | 4c | 40m 4s | 39m 48s | 40m 16s | 39m 43s |
| i686-gnu | 8c | | 1h 13m 38s | 1h 13m 39s | 1h 13m 48s | 1h 13m 12s |
| i686-gnu-nopt | 8c | | 1h 17m 44s | 1h 18m 14s | 1h 19m 55s | 1h 18m 44s |
| mingw-check | 4c | | 28m 15s | 27m 39s | 28m 36s | 28m 38s |
| test-various | 8c | 4c | 37m 45s | 37m 17s | 38m 26s | 38m 11s |
| x86_64-gnu | 4c | | 1h 34m 1s | 1h 31m 51s | 1h 30m 35s | 1h 32m 53s |
| x86_64-gnu-stable | 4c | | 1h 28m 26s | 1h 28m 11s | 1h 29m 40s | 1h 46m 28s |
| x86_64-gnu-aux | 4c | | 1h 33m 32s | 1h 31m 57s | 1h 34m 8s | 1h 32m 57s |
| x86_64-gnu-integration | 8c | | 1h 22m 2s | 1h 20m 14s | 1h 19m 46s | 1h 21m 24s |
| x86_64-gnu-debug | 8c | 4c | 52m 41s | 53m 40s | 51m 51s | 56m 9s |
| x86_64-gnu-distcheck | 8c | | 1h 9m 14s | 1h 5m 31s | 1h 6m 29s | 1h 5m 50s |
| x86_64-gnu-llvm-18 | 8c | | 1h 39m 47s | 1h 37m 57s | 1h 38m 40s | 1h 37m 38s |
| x86_64-gnu-llvm-17 | 8c | | 1h 41m 50s | 1h 45m 43s | 1h 45m 4s | 1h 43m 4s |
| x86_64-gnu-nopt | 4c | | 1h 20m 42s | 1h 21m 38s | 1h 20m 4s | 1h 22m 11s |
| x86_64-gnu-tools | 8c | | 1h 5m 0s | 1h 5m 30s | 1h 3m 1s | 1h 3m 20s |
| dist-x86_64-apple | xl | | 1h 35m 1s | 1h 39m 57s | 2h 2m 31s | 1h 47m 37s |
| dist-apple-various | xl | | 1h 18m 54s | 1h 22m 31s | 1h 13m 19s | 1h 38m 18s |
| x86_64-apple-1 | xl | | 1h 32m 8s | 1h 40m 12s | 1h 51m 28s | 1h 40m 26s |
| x86_64-apple-2 | xl | | 1h 0m 32s | 1h 4m 5s | 1h 9m 0s | 1h 7m 17s |
| dist-aarch64-apple | m1 | | 1h 3m 9s | 1h 1m 14s | 1h 2m 6s | 1h 2m 24s |
| aarch64-apple | m1 | | 53m 38s | 1h 1m 5s | 1h 3m 15s | 1h 6m 11s |
| x86_64-msvc | 8c | | 1h 27m 48s | 1h 29m 38s | 1h 29m 55s | 1h 28m 4s |
| i686-msvc | 8c | | 1h 38m 28s | 1h 34m 7s | 1h 39m 19s | 1h 39m 28s |
| x86_64-msvc-ext | 8c | | 1h 44m 5s | 1h 38m 40s | 1h 45m 21s | 1h 44m 19s |
| i686-mingw | 8c | | 1h 49m 57s | 1h 45m 1s | 1h 52m 4s | 1h 51m 4s |
| x86_64-mingw | 8c | | 1h 44m 2s | 1h 37m 36s | 1h 49m 58s | 1h 47m 5s |
| dist-x86_64-msvc | 8c | | 1h 57m 14s | 1h 49m 43s | 1h 52m 53s | 1h 52m 35s |
| dist-i686-msvc | 8c | | 1h 8m 5s | 1h 4m 9s | 1h 9m 26s | 1h 12m 0s |
| dist-aarch64-msvc | 8c | | 1h 18m 40s | 1h 14m 4s | 1h 22m 1s | 1h 19m 6s |
| dist-i686-mingw | 8c | | 1h 15m 36s | 1h 14m 36s | 1h 16m 38s | 1h 16m 2s |
| dist-x86_64-mingw | 8c | | 1h 11m 54s | 1h 16m 12s | 1h 16m 54s | 1h 18m 2s |
| dist-x86_64-msvc-alt | 8c | | 1h 11m 17s | 1h 10m 0s | 1h 11m 8s | 1h 13m 14s |
2024-05-20 11:30:58 +00:00
Dion Dokter
bd71c71ea0 Move tests to more opportune point 2024-05-20 12:50:37 +02:00
Dion Dokter
0d23a71ec2 Deduplicate test command 2024-05-20 12:18:24 +02:00
Dion Dokter
5c301429bd Only test std, alloc and core 2024-05-17 12:03:02 +02:00
Dion Dokter
b964b00668 Run tests with the flag enabled 2024-05-17 11:48:50 +02:00
Jakub Beránek
7b6a3d029b
CI: fix toolstate publishing 2024-05-15 15:48:52 +02:00
Matthias Krüger
0d4ae969eb
Rollup merge of #124829 - briansmith:b/armv7-profiler, r=Mark-Simulacrum
Enable profiler for armv7-unknown-linux-gnueabihf.

Allow code coverage measurement for armv7-unknown-linux-gnueabihf targets. Fixes #79640.
2024-05-11 23:43:24 +02:00
Daniel Paoliello
1b5d91b446 Reduce size of builders that take less than an hour 2024-05-10 14:09:39 -07:00
bors
a6e87c546d Auto merge of #124850 - dpaoliello:clang2022, r=Kobzol
Upgrade pre-built Clang used in MSVC and MacOS builds, move MSVC builds to Server 2022

Fixes #92948

Example working MacOS and Windows builds: <https://github.com/rust-lang/rust/actions/runs/8989360201>

There is a [bug in Clang 18](https://github.com/llvm/llvm-project/pull/81849) that causes issues when building for Arm64 in later parts of the build (specifically `libgit2`). As a workaround, we will still use the pre-built Clang to build LLVM but will use MSVC for the rest of the Arm64 build.
2024-05-10 00:04:22 +00:00
Daniel Paoliello
5212e07a4e Upgrade the version of Clang used in the build, move MSVC builds to Server 2022 2024-05-09 12:18:09 -07:00
bors
6f7e00a3e6 Auto merge of #124743 - Kobzol:arbitrary-try-build, r=pietroalbini
CI: enable arbitrary try builds, take two

Fixed version of https://github.com/rust-lang/rust/pull/124631, which hopefully won't completely break our CI this time 🤦‍♂️ Sorry once again. Only the last commit is new.

r? `@pietroalbini`
2024-05-09 10:39:50 +00:00
Marc Khouri
f44d611d85 Fuchsia test runner: fixup script
This commit fixes several issues in the fuchsia-test-runner.py script:

1. Migrate from `pm` to `ffx` for package management, as `pm` is now
deprecated. Furthermore, the `pm` calls used in this script no longer
work at Fuchsia's HEAD. This is the largest change in this commit, and
impacts all steps around repository management (creation and
registration of the repo, as well as package publishing).

2. Allow for `libtest` to be either statically or dynamically linked.
The script assumed it was dynamically linked, but the current Rust
behavior at HEAD is to statically link it.

3. Minor cleanup to use `ffx --machine json` rather than string parsing.

4. Minor cleanup to the docs around the script.
2024-05-07 17:29:38 +00:00
Brian Smith
de9b3b788f Enable profiler for armv7-unknown-linux-gnueabihf. 2024-05-06 15:25:37 -07:00
Jakub Beránek
b3e9060178
CI: fix auto builds and make sure that we always have at least a single CI job 2024-05-05 14:06:23 +02:00
Jakub Beránek
bf8bcc4c2d
Address review comments 2024-05-05 14:06:23 +02:00
Jakub Beránek
e2e280610d
Make the regex more robust 2024-05-05 14:06:23 +02:00
Jakub Beránek
74dbe8ae9e
Parse try build CI job name from commit message 2024-05-05 14:06:23 +02:00
Jakub Beránek
6778ecf960
Use sum type for WorkflowRunType 2024-05-05 14:06:23 +02:00
Matthias Krüger
f4d0776b4f
Rollup merge of #124663 - Kobzol:docker-local-download, r=Mark-Simulacrum
Enable reusing CI Docker cache when running CI images locally

When running a CI image locally, e.g. using `DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux`, it can take a long time until the Docker image is built, which is annoying.

Since we now use proper Docker caching on CI, it should be possible to just `docker pull` the prebuilt image to reuse the cache. We didn't want to do this on CI, since our caching key isn't perfect and it's possible that we can miss some changes, but I think that for local usage it is fine (we could introduce some env. var. to force disable the image download, if needed).

r? `@Mark-Simulacrum`
2024-05-04 22:27:31 +02:00
bors
6d721dd0b8 Auto merge of #124562 - dpaoliello:mingwserver2022, r=Mark-Simulacrum
Use Server 2022 in CI for mingw jobs

From my [previous testing](https://github.com/rust-lang/rust/issues/92948#issuecomment-2077771347), there is nothing blocking using Server 2022 for the Windows MinGW jobs.

This change:
* Adds new `os` entries for Server 2022.
* Renamed the Server 2019 `os` entries (as we'll be removing these over time).
* Moves the mingw jobs to Server 2022 and leaves the msvc jobs on Server 2019.

Partially fixes #92948
2024-05-04 07:49:50 +00:00
Jakub Beránek
02f4ef7cde
Allow reusing CI Docker cache when running CI images locally 2024-05-03 15:52:00 +02:00
Jakub Beránek
0372065097
Calculate the Docker hash key also locally 2024-05-03 15:50:02 +02:00
Jakub Beránek
27d1d0a6a4
Do not include Docker version in caching key
We are now using Docker caching directly rather than downloading archives from S3, so it should not be needed.

This should unblock reusing the cache locally.
2024-05-03 15:49:15 +02:00
Jakub Beránek
22456309ea
CI: remove env-x86_64-apple-tests YAML anchor 2024-05-02 21:23:52 +02:00
Daniel Paoliello
bc294b857e Use Server 2022 in CI for mingw jobs 2024-04-30 09:50:55 -07:00
Jakub Beránek
b194d5ce44
Output run_type from the matrix calculation job 2024-04-29 21:33:17 +02:00
Jakub Beránek
24bf3594a5
Rename JobType to WorkflowRunType 2024-04-29 21:33:17 +02:00
Jakub Beránek
0b1ecf1d8d
Remove the expand-yaml-anchors tool 2024-04-29 21:33:17 +02:00
Jakub Beránek
1ca92c0857
Replace dynamically generated ci.yml file with the original template 2024-04-29 21:32:35 +02:00
Jakub Beránek
23341500a6
Unify outcome jobs 2024-04-29 21:28:52 +02:00
Jakub Beránek
a700897d5a
Add missing checkout step 2024-04-27 15:48:35 +02:00
Jakub Beránek
163ea3328b
Move TOOLSTATE_REPO to top-level env 2024-04-27 11:36:42 +02:00
Jakub Beránek
e82c28debc
Remove master push hook 2024-04-25 11:06:27 +02:00
Jakub Beránek
86da0e758f
Remove useless condition
Neither `env.DEPLOY` nor `env.DEPLOY_ALT` should be present in this job.
2024-04-25 10:47:19 +02:00
Jakub Beránek
744dc2aa1d
Inline base-ci-job YAML anchor and remove unused anchors 2024-04-25 10:40:09 +02:00
Jakub Beránek
144bdf5bf1
Remove master CI job 2024-04-25 10:40:09 +02:00
Matthias Krüger
036bf570ad
Rollup merge of #124327 - Kobzol:ci-skip-jobs, r=pietroalbini
CI: implement job skipping in Python matrix calculation

This removes the `step` YAML anchor and the corresponding bash script.

Best reviewed commit-by-commit.

r? ```@pietroalbini```
2024-04-25 06:31:04 +02:00
León Orell Valerian Liehr
bef0f3fb4c
Rollup merge of #123881 - erickt:bump-fuchsia, r=tmandry
Bump Fuchsia versions

This updates the Fuchsia commit used in `auto - x86_64-gnu-integration` CI bot to use the Rust commit 703dc9ce64. This should help improve the coverage of this builder.

It also updates the SDK version to F20.20240412.3.1, and the Fuchsia Clang version to c777c011a709dffd4fa5e79cad7947b7c3405d02.

r? ``@tmandry``
2024-04-24 14:00:56 +02:00
Jakub Beránek
c36d78ed64
Turn CI_ONLY_WHEN_CHANNEL from an environment variable to a job attribute 2024-04-24 12:44:13 +02:00
Jakub Beránek
c59bc9a06a
Fix documentation 2024-04-24 12:15:17 +02:00
Jakub Beránek
4942a35f24
Remove step YAML anchor and env.SKIP_JOB 2024-04-24 11:12:39 +02:00
Jakub Beránek
246ee53637
Remove should-skip-this.sh 2024-04-24 11:12:17 +02:00
Jakub Beránek
e25735908f
Skip jobs based on the active channel in Python 2024-04-24 11:11:16 +02:00
Jakub Beránek
2a7fe14acd
Use a type alias for a CI job 2024-04-24 11:09:42 +02:00
bors
e7da0fa62f Auto merge of #124175 - Kobzol:ci-dynamic-job, r=pietroalbini
CI: dynamic jobs

This PR modifies our CI workflows to be dynamic. This means that when a GitHub event is generated, we will run a Python script (`calculate-job-matrix.py`), which decides which CI jobs should be generated. These jobs are defined in `src/ci/github-actions/jobs.yml`).

This should provide a few benefits:
- Once the migration to dynamic jobs is complete, we shouldn't need `expand-yaml-anchors` anymore.
- The job table on PRs (and also the left job column on auto/try builds) should be much cleaner and contain only the jobs that are actually relevant/executed.
- It should be much easier to support dynamic try builds, i.e. to run an arbitrary CI job on a try build.

See [this Zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/job.20matrix.20re-ordered.20PR.20list) for more context.

r? `@ghost`
2024-04-24 06:40:14 +00:00
Jakub Beránek
2632c6139d
Fix empty job matrix 2024-04-23 09:01:51 +02:00
Jakub Beránek
162892b030
Make separate envs for try and auto builds 2024-04-23 08:55:35 +02:00
Jakub Beránek
2f2d59dd06
Fix image field in auto jobs 2024-04-23 08:53:07 +02:00