Make it possible to build GCC on CI
This is the first step towards eventually enabling download of precompiled GCC from our CI.
Currently, we prebuild `libgccjit` on CI and cache it in Docker. This PR improves the bootstrap GCC step to make it work on CI, and also to make it faster by using sccache. After this change, an actual build on CI should take only 2-3 minutes.
Note that this PR does not yet remove the `build-gccjit.sh` script and replace it with the bootstrap step, I'll leave that to a follow-up PR.
The added `flex` package and the ZSTD library fix were needed to make GCC build on CI.
CC ``````@GuillaumeGomez``````
r? ``````@onur-ozkan``````
Do not include GCC source code in source tarballs
The licensing story is unclear, it makes the archive much larger, and we should not need it for building anything in the tarballs (yet).
```
Before:
121s building the archive
1.3 GiB gzipped size
5.7 GiB extracted size
402519 extracted files
After:
64s building the archive
961 MiB gzipped size
4.5 GiB extracted size
257719 extracfed files
```
Fixes: https://github.com/rust-lang/rust/issues/135606
r? `@ehuss`
Add Profile Override for Non-Git Sources
## PR description
- Fixes#135358
This PR introduces the following updates to
1. `bootstrap.py`:
- If the `profile` is `None` and the source is non-git, the `profile` is automatically overridden to `"dist"`.
- Ensures that options like `download-ci-llvm` and `download-rustc` are not used with non-git sources. An exception is raised if these options are present in the configuration when the source is non-git.
2. `bootstrap_test.py`
- Added unit tests to verify both the profile override mechanism and the assertion for restricted options.
These tests ensure the correct behavior for non-git sources and the handling of `if-unchanged` options.
r? `@onur-ozkan`
`@rustbot` T-bootstrap
make it possible to use ci-rustc on tarball sources
Previously, bootstrap was using `Config::last_modified_commit` unconditionally to figure the commit to download precompiled rustc artifact from CI, which was leading builds to fail on tarball sources as `Config::last_modified_commit` requires `git` to be present in the project source. This change makes bootstrap to call `Config::last_modified_commit` only when it's running on git-managed source and read `git-commit-hash` file otherwise.
Previously, bootstrap was using `Config::last_modified_commit` unconditionally to figure
the commit has to download precompiled rustc artifact from CI, which was leading builds to
fail on tarball sources as `Config::last_modified_commit` requires `git` to be present in the project
source. This change makes bootstrap to call `Config::last_modified_commit` only when it's running on
git-managed source and read `git-commit-hash` file otherwise.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Rollup of 7 pull requests
Successful merges:
- #133700 (const-eval: detect more pointers as definitely not-null)
- #135290 (Encode constraints that hold at all points as logical edges in location-sensitive polonius)
- #135478 (Run clippy for rustc_codegen_gcc on CI)
- #135583 (Move `std::pipe::*` into `std::io`)
- #135612 (Include x scripts in tarballs)
- #135624 (ci: mirror buildkit image to ghcr)
- #135661 (Stabilize `float_next_up_down`)
r? `@ghost`
`@rustbot` modify labels: rollup
bootstrap: still require `COMPILETEST_FORCE_STAGE0` for `./x test rustdoc-js --stage 0`
This PR reverts #135375, because through some more testing I found out `./x test rustdoc-js --stage 0` does not in fact build rustdoc, and all the tests fail. This can't be intended behavior, so at least require `COMPILETEST_FORCE_STAGE0` to make it less likely to run `rustdoc-js --stage 0` by accident.
The problem that `--stage 0` is not working at all for this rustdoc-js test suite is tracked over at #135603.
cc `@lolbinarycat`
r? bootstrap
Add license-metadata.json to rustc-src tarball.
Adds a license-metadata.json to the source tarball.
This file was reported as missing as a comment on #133461, and it prevents you building the compiler from the source tarball (unless you re-generate it yourself, which is non-obvious and requires `reuse` to be installed).
r? Kobzol
resolve symlinks of LLVM tool binaries before copying them
There is a chance that these tools are being installed from an external LLVM and we have no control over them. If any of these tools use symlinks, they will fail during tarball distribution. This change makes copying process to resolve symlinks just before placing them into the destination path.
Fixes https://github.com/rust-lang/rust/issues/135554
There is a chance that these tools are being installed from an external LLVM
and we have no control over them. If any of these tools use symlinks, they will
fail during tarball distribution. This change makes copying process to resolve
symlinks just before placing them into the destination path.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Add COPYRIGHT-*.html files to distribution and update `COPYRIGHT`
* Updates the `COPYRIGHT` file to describe how we actually do things now, and removes the licence text from it as they are stored elsewhere.
* dist tarballs get all of the files in `LICENSES/*`.
* This folder is managed by `reuse` and each file exists because we refer to the licence somewhere in our tree. We should be supplying these licence texts to anyone who obtains a copy of the source code and now we do.
* The binary rust tarball gets `COPYRIGHT.html` and `COPYRIGHT-library.html`, which are auto-generated files that describe the licence information for both the in-tree source files used to build the Rust toolchain, and the out-of-tree dependencies we used to build the toolchain.
* The other binary tarballs are unchanged, for now. In future you need to make a call whether to ship multiple version of COPYRIGHT.html, or whether to try and make, for example, a cargo-specific COPYRIGHT.html file.
* The `LICENSE-MIT` file now includes a blanket copyright statement, as the text indicates that it should and because users will expect to know who owns the copyright of the material they have been given (even if the answer is 'lots of people').
try-job: x86_64-fuchsia
bootstrap: Implement conditional `tracing` infra
Add a conditional `tracing` setup that is gated behind `BOOTSTRAP_TRACING` env var. This `tracing` infra is implemented by:
- Introducing an optional `tracing` cargo feature in bootstrap.
- Added optional `tracing*` dependencies which are gated behind the `tracing` cargo feature.
- When `BOOTSTRAP_TRACING` is set, `bootstrap.py` will build bootstrap with `--features=tracing`.
There is a small trick here to share `BOOTSTRAP_TRACING` env var without having to add a separate env var:
- `BOOTSTRAP_TRACING=1` is not a registered `tracing` filter target, so that can be used to enable the `tracing` cargo feature yet not actually enable any tracing logs (useful for editor r-a setups without actually outputting any tracing logs).
- `BOOTSTRAP_TRACING=TRACE` and such are actually valid `tracing` filters, but that sets `BOOTSTRAP_TRACING` anyway.
Example usage: https://github.com/rust-lang/rust/pull/135299 (that experimental PR is not conditionally gated)
This PR is intentionally kept minimal to focus on the infra itself. To get actual mileage, instrumentations will need to be added to individual `Step`s and such.
r? `@onur-ozkan` (or reroll)
