This commit builds on #65501 continue to simplify the build system and
compiler now that we no longer have multiple LLVM backends to ship by
default. Here this switches the compiler back to what it once was long
long ago, which is linking LLVM directly to the compiler rather than
dynamically loading it at runtime. The `codegen-backends` directory of
the sysroot no longer exists and all relevant support in the build
system is removed. Note that `rustc` still supports a dynamically loaded
codegen backend as it did previously, it just no longer supports
dynamically loaded codegen backends in its own sysroot.
Additionally as part of this the `librustc_codegen_llvm` crate now once
again explicitly depends on all of its crates instead of implicitly
loading them through the sysroot. This involved filling out its
`Cargo.toml` and deleting all the now-unnecessary `extern crate`
annotations in the header of the crate. (this in turn required adding a
number of imports for names of macros too).
The end results of this change are:
* Rustbuild's build process for the compiler as all the "oh don't forget
the codegen backend" checks can be easily removed.
* Building `rustc_codegen_llvm` is much simpler since it's simply
another compiler crate.
* Managing the dependencies of `rustc_codegen_llvm` is much simpler since
it's "just another `Cargo.toml` to edit"
* The build process should be a smidge faster because there's more
parallelism in the main rustc build step rather than splitting
`librustc_codegen_llvm` out to its own step.
* The compiler is expected to be slightly faster by default because the
codegen backend does not need to be dynamically loaded.
* Disabling LLVM as part of rustbuild is still supported, supporting
multiple codegen backends is still supported, and dynamic loading of a
codegen backend is still supported.
With #65251 landed there's no need to build two LLVM backends and ship
them with rustc, every target we have now uses the same LLVM backend!
This removes the `src/llvm-emscripten` submodule and additionally
removes all support from rustbuild for building the emscripten LLVM
backend. Multiple codegen backend support is left in place for now, and
this is intended to be an easy 10-15 minute win on CI times by avoiding
having to build LLVM twice.
- Compatible with Emscripten 1.38.46-upstream or later upstream.
- Refactors the Emscripten target spec to share code with other wasm
targets.
- Replaces the old incorrect wasm32 C call ABI with the correct one,
preserving the old one as wasm32_bindgen_compat for wasm-bindgen
compatibility.
- Updates the varargs ABI used by Emscripten and deletes the old one.
- Removes the obsolete wasm32-experimental-emscripten target.
- Uses EMCC_CFLAGS on CI to avoid the timeout problems with #63649.
- Refactors the Emscripten target spec to share code with other wasm
targets.
- Replaces the incorrect wasm32 C call ABI with the old asmjs
version, which is correct for both wasm32 and JS.
- Updates the varargs ABI used by Emscripten and deletes the old one.
- Removes the obsolete wasm32-experimental-emscripten target.
- Temporarily makes Emscripten targets use panic=abort by default
because supporting unwinding will require an LLVM patch.
This commit changes the return type of `Builder::cargo` to return a
builder that allows dynamically adding more `RUSTFLAGS` values
after-the-fact. While not used yet, this will later be used to delete
more of `rustc.rs`
This commit removes the `wasm_syscall` feature from the
wasm32-unknown-unknown build of the standard library. This feature was
originally intended to allow an opt-in way to interact with the
operating system in a posix-like way but it was never stabilized.
Nowadays with the advent of the `wasm32-wasi` target that should
entirely replace the intentions of the `wasm_syscall` feature.
Since its inception rustbuild has always worked in three stages: one for
libstd, one for libtest, and one for rustc. These three stages were
architected around crates.io dependencies, where rustc wants to depend
on crates.io crates but said crates don't explicitly depend on libstd,
requiring a sysroot assembly step in the middle. This same logic was
applied for libtest where libtest wants to depend on crates.io crates
(`getopts`) but `getopts` didn't say that it depended on std, so it
needed `std` built ahead of time.
Lots of time has passed since the inception of rustbuild, however,
and we've since gotten to the point where even `std` itself is depending
on crates.io crates (albeit with some wonky configuration). This
commit applies the same logic to the two dependencies that the `test`
crate pulls in from crates.io, `getopts` and `unicode-width`. Over the
many years since rustbuild's inception `unicode-width` was the only
dependency picked up by the `test` crate, so the extra configuration
necessary to get crates building in this crate graph is unlikely to be
too much of a burden on developers.
After this patch it means that there are now only two build phasese of
rustbuild, one for libstd and one for rustc. The libtest/libproc_macro
build phase is all lumped into one now with `std`.
This was originally motivated by rust-lang/cargo#7216 where Cargo was
having to deal with synthesizing dependency edges but this commit makes
them explicit in this repository.
This is no longer used by the index generator and was always an unstable
compiler detail, so strip it out.
This also leaves in RUSTC_ERROR_METADATA_DST since the stage0 compiler
still needs it to be set.
rustc: Stabilize options for pipelined compilation
This commit stabilizes options in the compiler necessary for Cargo to
enable "pipelined compilation" by default. The concept of pipelined
compilation, how it's implemented, and what it means for rustc are
documented in #60988. This PR is coupled with a PR against Cargo
(rust-lang/cargo#7143) which updates Cargo's support for pipelined
compliation to rustc, and also enables support by default in Cargo.
(note that the Cargo PR cannot land until this one against rustc lands).
The technical changes performed here were to stabilize the functionality
proposed in #60419 and #60987, the underlying pieces to enable pipelined
compilation support in Cargo. The issues have had some discussion during
stabilization, but the newly stabilized surface area here is:
* A new `--json` flag was added to the compiler.
* The `--json` flag can be passed multiple times.
* The value of the `--json` flag is a comma-separated list of
directives.
* The `--json` flag cannot be combined with `--color`
* The `--json` flag must be combined with `--error-format=json`
* The acceptable list of directives to `--json` are:
* `diagnostic-short` - the `rendered` field of diagnostics will have a
"short" rendering matching `--error-format=short`
* `diagnostic-rendered-ansi` - the `rendered` field of diagnostics
will be colorized with ansi color codes embedded in the string field
* `artifacts` - JSON blobs will be emitted for artifacts being emitted
by the compiler
The unstable `-Z emit-artifact-notifications` and `--json-rendered`
flags have also been removed during this commit as well.
Closes#60419Closes#60987Closes#60988
This commit stabilizes options in the compiler necessary for Cargo to
enable "pipelined compilation" by default. The concept of pipelined
compilation, how it's implemented, and what it means for rustc are
documented in #60988. This PR is coupled with a PR against Cargo
(rust-lang/cargo#7143) which updates Cargo's support for pipelined
compliation to rustc, and also enables support by default in Cargo.
(note that the Cargo PR cannot land until this one against rustc lands).
The technical changes performed here were to stabilize the functionality
proposed in #60419 and #60987, the underlying pieces to enable pipelined
compilation support in Cargo. The issues have had some discussion during
stabilization, but the newly stabilized surface area here is:
* A new `--json` flag was added to the compiler.
* The `--json` flag can be passed multiple times.
* The value of the `--json` flag is a comma-separated list of
directives.
* The `--json` flag cannot be combined with `--color`
* The `--json` flag must be combined with `--error-format=json`
* The acceptable list of directives to `--json` are:
* `diagnostic-short` - the `rendered` field of diagnostics will have a
"short" rendering matching `--error-format=short`
* `diagnostic-rendered-ansi` - the `rendered` field of diagnostics
will be colorized with ansi color codes embedded in the string field
* `artifacts` - JSON blobs will be emitted for artifacts being emitted
by the compiler
The unstable `-Z emit-artifact-notifications` and `--json-rendered`
flags have also been removed during this commit as well.
Closes#60419Closes#60987Closes#60988
Now that we've fully moved to Azure Pipelines and bors has been updated
to only gate on Azure this commit removes the remaining Travis/AppVeyor
support contained in this repository. Most of the deletions here are
related to producing better output on Travis by folding certain
sections. This isn't supported by Azure so there's no need to keep it
around, and if Azure ever adds support we can always add it back!
ci: Attempt to skip a full rustc compile on dist*
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
This commit furthers the previous one to ensure that we don't build an
extra stage of the compiler in CI. A test has been added to rustbuild to
ensure that this doesn't regress, and then in debugging this test it was
hunted down that the `dist::Std` target was the one erroneously pulling
in the wrong compiler.
The `dist::Std` step was updated to instead account for the "full
bootstrap" or not flag, ensuring that the correct compiler for compiling
the final standard library was used. This was another use of the
`force_use_stage1` function which was in theory supposed to be pretty
central, so existing users were all evaluated and a new function,
`Builder::compiler_for`, was introduced. All existing users of
`force_use_stage1` have been updated to use `compiler_for`, where the
semantics of `compiler_for` are similar to that of `compiler` except
that it doesn't guarantee the presence of a sysroot for the arguments
passed (as they may be modified).
Perhaps one day we can unify `compiler` and `compiler_for`, but the
usage of `Builder::compiler` is so ubiquitous it would take quite some
time to evaluate whether each one needs the sysroot or not, so it's
hoped that can be done in parallel.
Add basic CDB support to debuginfo compiletest s, to help catch `*.natvis` regressions, like those fixed in #60687.
First draft, feedback welcome.
Several Microsoft debuggers (VS, VS Code, WinDbg, CDB, ...) consume the `*.natvis` files we embed into rust `*.pdb` files. While this only tests CDB, that test coverage should help for all of them.
# Changes
## src\bootstrap
- test.rs: Run CDB debuginfo tests on MSVC targets
## src\test\debuginfo
- issue-13213.rs: CDB has trouble with this, skip for now (newly discovered regression?)
- pretty-std.rs: Was ignored, re-enable for CDB only to start with, add CDB tests.
- should-fail.rs: Add CDB tests.
## src\tools\compiletest:
- Added "-cdb" option
- Added Mode::DebugInfoCdb ("debuginfo-cdb")
- Added run_debuginfo_cdb_test[_no_opt]
- Renamed Mode::DebugInfoBoth -> DebugInfoGdbLldb ("debuginfo-gdb+lldb") since it's no longer clear what "Both" means.
- Find CDB at the default Win10 SDK install path "C:\Program Files (x86)\Windows Kits\10\Debugger\\*\cdb.exe"
- Ignore CDB tests if CDB not found.
# Issues
- `compute_stamp_hash`: not sure if there's any point in hashing `%ProgramFiles(x86)%`
- `OsString` lacks any `*.natvis` entries (would be nice to add in a followup changelist)
- DSTs (array/string slices) which work in VS & VS Code fail in CDB.
- I've avoided `Mode::DebugInfoAll` as 3 debuggers leads to pow(2,3)=8 possible combinations.
# Reference
CDB is not part of the base Visual Studio install, but can be added via the Windows 10 SDK:
https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk
Installing just "Debugging Tools for Windows" is sufficient.
CDB appears to already be installed on appveyor CI, where this changelist can find it, based on it's use here:
0ffc573110/appveyor.yml (L227)
CDB commands and command line reference:
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-reference
Several Microsoft debuggers (VS, VS Code, WinDbg, CDB, ...) consume the `*.natvis` files we embed into rust `*.pdb` files.
While this only tests CDB, that test coverage should help for all of them.
CHANGES
src\bootstrap
- test.rs: Run CDB debuginfo tests on MSVC targets
src\test\debuginfo
- issue-13213.rs: CDB has trouble with this, skip for now (newly discovered regression?)
- pretty-std.rs: Was ignored, re-enable for CDB only to start with, add CDB tests.
- should-fail.rs: Add CDB tests.
src\tools\compiletest:
- Added "-cdb" option
- Added Mode::DebugInfoCdb ("debuginfo-cdb")
- Added run_debuginfo_cdb_test[_no_opt]
- Renamed Mode::DebugInfoBoth -> DebugInfoGdbLldb ("debuginfo-gdb+lldb") since it's no longer clear what "Both" means.
- Find CDB at the default Win10 SDK install path "C:\Program Files (x86)\Windows Kits\10\Debugger\*\cdb.exe"
- Ignore CDB tests if CDB not found.
ISSUES
- `compute_stamp_hash`: not sure if there's any point in hashing `%ProgramFiles(x86)%`
- `OsString` lacks any `*.natvis` entries (would be nice to add in a followup changelist)
- DSTs (array/string slices) which work in VS & VS Code fail in CDB.
- I've avoided `Mode::DebugInfoAll` as 3 debuggers leads to pow(2,3)=8 possible combinations.
REFERENCE
CDB is not part of the base Visual Studio install, but can be added via the Windows 10 SDK:
https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk
Installing just "Debugging Tools for Windows" is sufficient.
CDB appears to already be installed on appveyor CI, where this changelist can find it, based on it's use here:
0ffc573110/appveyor.yml (L227)
CDB commands and command line reference:
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-reference
Allow subdirectories to be tested by x.py test
Fixes https://github.com/rust-lang/rust/issues/60718.
As far as I can tell, multiple `--test-args` flags are ignored (only the first is respected), so if you specify a subdirectory, you won't also be able to filter using `--test-args`. If you don't specify a subdirectory, `--test-args` will continue working as usual, so this is strictly an improvement on the current state of affairs.
