Move the standard library to a separate workspace
This ensures that the Cargo.lock packaged for it in the rust-src component is up-to-date, allowing rust-analyzer to run cargo metadata on the standard library even when the rust-src component is stored in a read-only location as is necessary for loading crates.io dependencies of the standard library.
This also simplifies tidy's license check for runtime dependencies as it can now look at all entries in library/Cargo.lock without having to filter for just the dependencies of runtime crates. In addition this allows removing an exception in check_runtime_license_exceptions that was necessary due to the compiler enabling a feature on the object crate which pulls in a dependency not allowed for the standard library.
While cargo workspaces normally enable dependencies of multiple targets to be reused, for the standard library we do not want this reusing to prevent conflicts between dependencies of the sysroot and of tools that are built using this sysroot. For this reason we already use an unstable cargo feature to ensure that any dependencies which would otherwise be shared get a different -Cmetadata argument as well as using separate build dirs.
This doesn't change the situation around vendoring. We already have several cargo workspaces that need to be vendored. Adding another one doesn't change much.
There are also no cargo profiles that are shared between the root workspace and the library workspace anyway, so it doesn't add any extra work when changing cargo profiles.
allow setting `link-shared` and `static-libstdcpp` with CI LLVM
These options also affect `compiler/rustc_llvm` builds. They should be configurable even when using CI LLVM.
r? ```@cuviper```
bootstrap: fix bug preventing the use of custom targets
the bug was caused by two factors:
1. only checking the RUST_TARGET_PATH form, not the full filepath form
2. indirectly trying to use the Debug presentation to get the file path
These options also affect `compiler/rustc_llvm` builds. They should be configurable
even when using CI LLVM.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
the bug was caused by two factors:
1. only checking the RUST_TARGET_PATH form, not the full filepath form
2. indirectly trying to use the Debug presentation to get the file path
This ensures that the Cargo.lock packaged for it in the rust-src
component is up-to-date, allowing rust-analyzer to run cargo metadata on
the standard library even when the rust-src component is stored in a
read-only location as is necessary for loading crates.io dependencies of
the standard library.
This also simplifies tidy's license check for runtime dependencies as it
can now look at all entries in library/Cargo.lock without having to
filter for just the dependencies of runtime crates. In addition this
allows removing an exception in check_runtime_license_exceptions that
was necessary due to the compiler enabling a feature on the object crate
which pulls in a dependency not allowed for the standard library.
While cargo workspaces normally enable dependencies of multiple targets
to be reused, for the standard library we do not want this reusing to
prevent conflicts between dependencies of the sysroot and of tools that
are built using this sysroot. For this reason we already use an unstable
cargo feature to ensure that any dependencies which would otherwise be
shared get a different -Cmetadata argument as well as using separate
build dirs.
This doesn't change the situation around vendoring. We already have
several cargo workspaces that need to be vendored. Adding another one
doesn't change much.
There are also no cargo profiles that are shared between the root
workspace and the library workspace anyway, so it doesn't add any extra
work when changing cargo profiles.
Update sysinfo version to 0.31.2
I needed to update `memchr` version (which was pinned in 36a16798f7). So let's see if it triggers the linker issue.
try-job: x86_64-mingw
improve bootstrap to allow selecting llvm tools individually
Everything works as before, + now bootstrap allows for individually selecting LLVM tools (e.g., `x dist opt llvm-dis`) to include in the dist artifact.
Everything works as before, + now bootstrap allows for individually selecting LLVM
tools (e.g., `x dist opt llvm-dis`) to include in the dist artifact.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
This allows the standard library to be compiled even with `download-rustc` enabled.
Which means it's no longer a requirement to compile `rustc` in order to compile `std`.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
`crates` field is handled in the `Step::make_run` just like in any other
`Std` implementation, so we don't need to resolve them in `Std::new`.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Since we now handle library crates properly, there's no need to panic for `no_std`
targets anymore.
`x doc library` now generates documentation for the `alloc` crate from standard library.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
This change unifies the `Step::run_make` logic and improves it by skipping
std specific crates for no_std targets.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Update the reference
This updates the reference to use the new mdbook-spec preprocessor, which is a Cargo library inside the reference submodule.
Note that this PR contains a bunch of bootstrap cleanup commits to assist with making sure the submodules are working correctly. All of the cleanup PRs should have a description in their commit. I'd be happy to move those to a separate PR if that makes review easier.
The main changes for the reference are:
- Move the `doc::Reference` bootstrap step out of the generic macro into a custom step.
- This step needs to build rustdoc because the new mdbook-spec plugin uses rustdoc for generating links.
- PATH is updated so that the rustdoc binary can be found.
- rustbook now includes the mdbook-spec plugin as a dependency.
- rustbook enables the mdbook-spec preprocessor.
I did a bunch of testing with the various commands and setups, such as:
- `submodules=true` and `submodules=false`
- having all submodules deinitialized
- not in a git repository
However, there are probably thousands of different permutations of different commands, settings, and environments, so there is a chance I'm missing something.
Bootstrap command refactoring: make command output API more bulletproof (step 7)
Continuation of https://github.com/rust-lang/rust/pull/127680.
This PR modifies the API of running commands to make it more explicit when a command is expected to produce programmatically handled output. Now if you call just `run`, you cannot access the stdout/stderr by accident, because it will not be returned to the caller.
This API change might be seen as overkill, let me know what do you think. In any case, I'd like to land the second commit, to make it harder to accidentally read stdout/stderr of commands that did not capture output (now you'd get an empty string as a result, but you should probably get a panic instead, if you try to read uncaptured stdout/stderr).
Tracking issue: https://github.com/rust-lang/rust/issues/126819
r? `@onur-ozkan`
try-job: x86_64-msvc
I misread this one. It is only checking if LLVM needs to be rebuilt.
There is code below that handles the case where it is unable to compute
the stamp if the source is missing.
These are required 100% of the time, but they are almost always required
for any command that runs Cargo in the main workspace.
Ideally, initializing these two standard library submodules would be
lazy and only initialized when required (see
https://github.com/rust-lang/rust/pull/82653). However, it would require
updating these in almost every Step (anything that runs `cargo` in the
main workspace).
If e.g. only stdout is captured, but the caller tries to read stderr, previously
they would get back an empty string. Now the code will explicitly panic when
accessing an uncaptured output stream.
Now there are separate functions for running a command without capturing, running while capturing stdout
and running while capturing everything. This should help avoid situations where stdout/stderr is accessed
when it was not captured.
We were running testing on API 18, which was already out of support for
NDK 25, and some of the ancient behavior in that image was causing
trouble when developing `rustc` features (#120326).
Update to the current LTS NDK 26, and to its minimum supported API 21.
Fixes: #120567
This updates the reference which is now using a new mdbook plugin. This
requires a little extra work than a normal book because the plugin uses
`rustdoc` to generate links to the standard library. It also ensures
that the submodule is available for *any* command that uses rustbook,
since it is now part of the rustbook workspace.
