Replace black with ruff in `tidy`
`ruff` can both lint and format Python code (in fact, it should be a mostly drop-in replacement for `black` in terms of formatting), so it's not needed to use `black` anymore. This PR removes `black` and replaces it with `ruff`, to get rid of one Python dependency, and also to make Python formatting faster (although that's a small thing).
If we decide to merge this, we'll need to "reformat the world" - `ruff` is not perfectly compatible with `black`, and it also looks like `black` was actually ignoring some files before. I tried it locally (`./x test tidy --extra-checks=py:fmt --bless`) and it also reformatted some code in subtrees (e.g. `clippy` or `rustc_codegen_gcc`) - I'm not sure how to handle that.
explain the options bootstrap passes to curl
also fixes a discrepancy where the rust side doesn't use -L
docs are only on the rust side, since duplicated prose has a tendancy to get out-of-sync, and also because there are talks of removing the python script all together eventually.
also fixes a discrepency where the rust side doesn't use -L
must not be merged before #129134
docs are only on the rust side, since duplicated prose
has a tendancy to get out-of-sync, and also because there
are talks of removing the python script all together eventually.
handle stage0 `cargo` and `rustc` separately
This change allows setting either `build.cargo` or `build.rustc` without requiring both to be set simultaneously, which was not possible previously.
To try it, set `build.rustc` without setting `build.cargo`, and try to bootstrap on clean build.
Blocker for https://github.com/rust-lang/rust/pull/129152
This change allows setting either `build.cargo` or `build.rustc` without requiring
both to be set simultaneously, which was not possible previously.
To try it, set `build.rustc` without setting `build.cargo`, and try to bootstrap on clean build.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Technically, wiping bootstrap builds can increase the build time.
But in practice, trying to manually resolve post-bump issues and
even accidentally removing the entire build directory will result
in a much greater loss of time. After all, the bootstrap build process
is not a particularly lengthy operation.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
LLVM 18 now ships `libLLVM*.so.*`, so `.so` is not the sole extension
anymore, which breaks the dylib detection. Oops! Adjust it to only
search for `.so` somewhere.
Setting the bootstrap cache path to an external location can help to
speed up builds in cases where the build directory is not kept between
builds, e.g. in CI or other automated build systems.
This allows the bootstrap cache path to be set by the
`RUSTC_BOOTSTRAP_CACHE` environment variable.
Setting the bootstrap cache path to an external location can help to
speed up builds in cases where the build directory is not kept between
builds, e.g. in CI or other automated build systems.
before this change, an incorrect profile would result in the following error:
```sh
...
...
File "/home/nimda/devspace/onur-ozkan/rust/src/bootstrap/bootstrap.py", line 1088, in bootstrap
with open(include_path) as included_toml:
^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/nimda/devspace/onur-ozkan/rust/src/bootstrap/defaults/config.aaaa.toml'
```
with this change, the error message is now:
```sh
...
...
File "/home/nimda/devspace/onur-ozkan/rust/src/bootstrap/bootstrap.py", line 1088, in bootstrap
raise Exception("Unrecognized profile '{}'. Check src/bootstrap/defaults"
Exception: Unrecognized profile 'aaaa'. Check src/bootstrap/defaults for available options.
```
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Note that RUSTFLAGS_BOOTSTRAP should always be added to the end of
RUSTFLAGS to be actually effective (e.g., if we have `-Dwarnings` in
RUSTFLAGS, passing `-Awarnings` from RUSTFLAGS_BOOTSTRAP should override it).
Signed-off-by: onur-ozkan <work@onurozkan.dev>
warn if source is not either a git clone or a dist tarball
When the repository is downloaded directly via HTTP(as in #115041), builds may fail due to missing submodules.
This PR adds a check that warns people in such cases.
avoid stdout redirection on `curl` executions
Avoid redirecting the curl output directly to the stdout. This alteration affects the integrity of the file during the retry process, as it also redirects the logs from the retries. Consequently, this leads to the bootstrap process failing because of an invalid checksum.
For more information, see the [zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/checksum.20errors)
Fixes#115275
Avoid redirecting the curl output directly to the stdout. This alteration
affects the integrity of the file during the retry process, as it also redirects
the logs from the retries. Consequently, this leads to the bootstrap process failing
because of an invalid checksum.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
In addition: Incorporated some review feedback (namely, removed a useless
initial assignment to True that was never read), and unified code a bit more
between bootstrap.py and download.rs (by using the same variable name for the
same concept).
1. Turned patch-binaries-for-nix from a boolean into a ternary flag: true,
false, and unset.
2. When patch-binaries-for-nix is unset, we continue with the existing NixOS
detection heuristic (look for nixos in /etc/os-release, if present), but if
we are not atop NixOS, then issue a note if we see the IN_NIX_SHELL
environment variable telling the user to consider setting
patch-binaries-for-nix explicitly.
[Two users over on zulip](https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Bootstrapping.20on.20NixOS) bumped into issues where NixOS wasn't being properly detected.
I believe this was caused by the presence of `/lib` on their machines. `/lib` is not standard on NixOS but can still be created by users or scripts.
We are already checking `/etc/os-release`. The presence of `ID=nixos` in it's output should be trustworthy and we shouldn't then go on to also check for `/lib`.
