Commit graph

4073 commits

Author SHA1 Message Date
David Wood
241160de72
bootstrap: copy llvm-dwp to sysroot
`llvm-dwp` is required for linking the DWARF objects into DWARF packages
when using Split DWARF, especially given that rustc produces multiple
DWARF objects (one for each codegen unit).

Signed-off-by: David Wood <david@davidtw.co>
2020-12-16 10:33:45 +00:00
Yuki Okushi
d90084c226
Rollup merge of #79970 - bjorn3:no_unnecessary_llvm_checkout, r=Mark-Simulacrum
Misc rustbuild improvements when the LLVM backend isn't used

* Don't checkout llvm-project
* Don't require cmake and ninja

Fixes #78564
2020-12-13 11:05:43 +09:00
Yuki Okushi
54119d4436
Rollup merge of #79954 - jyn514:normalize-oops, r=Mark-Simulacrum
Fix building compiler docs with stage 0

This regressed in https://github.com/rust-lang/rust/pull/79525 (see https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Can't.20document.20single.20crate).

r? `@Mark-Simulacrum`
2020-12-13 11:05:40 +09:00
Yuki Okushi
1b81f08d4c
Rollup merge of #79940 - matthiaskrgr:cl15ppy, r=Dylan-DPC
fix more clippy::complexity findings

fix clippy::unnecessary_filter_map
use if let Some(x) = ..  instead of ...map(|x|) to conditionally run fns that return () (clippy::option_map_unit_fn)
fix clippy::{needless_bool, manual_unwrap_or}
don't clone types that are copy (clippy::clone_on_copy)
don't convert types into identical types with .into() (clippy::useless_conversion)
use strip_prefix over slicing (clippy::manual_strip)

r? ``@Dylan-DPC``
2020-12-13 11:05:36 +09:00
Yuki Okushi
f80c6aeb87
Rollup merge of #79845 - jyn514:python3, r=Mark-Simulacrum
Fix rustup support in default_build_triple for python3

bootstrap completely ignores all errors when detecting a rustup version,
so this wasn't noticed before.

Fixes the following error:

```
rustup not detected: a bytes-like object is required, not 'str'
falling back to auto-detect
```

This also takes the opportunity to only call rustup and other external
commands only once during startup.

Follow-up to https://github.com/rust-lang/rust/pull/78513.
2020-12-13 11:05:35 +09:00
Joshua Nelson
9df0348299 Fix building compiler docs with stage 0 2020-12-12 17:31:35 -05:00
bjorn3
d79e19f332 Don't require cmake and ninja when the LLVM backend is not used 2020-12-12 18:13:04 +01:00
bjorn3
16f69b5430 Don't checkout llvm-project when the LLVM backend isn't built 2020-12-12 18:06:07 +01:00
Matthias Krüger
b7795e135a fix clippy::{needless_bool, manual_unwrap_or} 2020-12-11 23:02:19 +01:00
Matthias Krüger
db6c50998c don't clone types that are copy (clippy::clone_on_copy) 2020-12-11 23:02:19 +01:00
Matthias Krüger
5c8de1cf49 use strip_prefix over slicing (clippy::manual_strip) 2020-12-11 23:02:17 +01:00
Corey Farwell
25bfa15371 Manually code-sign after running install_name_tool 2020-12-10 18:51:30 -05:00
Corey Farwell
d1911dd075 If "aarch64-apple-darwin", copy sanitizers. 2020-12-10 00:18:43 -05:00
Corey Farwell
5940c19315 Enable ASan, TSan, UBSan for aarch64-apple-darwin. 2020-12-09 23:53:53 -05:00
Joshua Nelson
8909c4dc31 Fix rustup support in default_build_triple for python3
bootstrap completely ignores all errors when detecting a rustup version,
so this wasn't noticed before.

Fixes the following error:

```
rustup not detected: a bytes-like object is required, not 'str'
falling back to auto-detect
```

This also takes the opportunity to only call rustup and other external
commands only once during startup.
2020-12-08 19:38:25 -05:00
Matthias Krüger
dbe3acfaeb don't wrap code block in Ok() (clipppy::unit_arg) 2020-12-08 20:27:48 +01:00
Nixon Enraght-Moony
1098cce27a Add tests for rustdoc json
Move rustdoc/rustdoc-json to rustdoc-json

Scaffold rustdoc-json test mode

Implement run_rustdoc_json_test

Fix up python

Make tidy happy
2020-12-01 18:34:39 +00:00
Mara Bos
f45e6953d7
Rollup merge of #79527 - jyn514:intra-doc-tests, r=Manishearth
Move intra-doc link tests into a subdirectory

They were starting to get unwieldy.

r? ``@Manishearth``
2020-12-01 10:50:19 +00:00
Mara Bos
99e075ff29
Rollup merge of #79525 - jyn514:feature-gate-normalize, r=GuillaumeGomez
Add -Z normalize-docs and enable it for compiler docs

Works around https://github.com/rust-lang/rust/issues/79459 by only enabling normalization for the compiler itself (and anyone who opts-in on nightly). Eventually I want to remove this and enable normalization by default, but that's turned out to be [really hard](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/How.20do.20I.20normalize.20projection.20types.20to.20a.20single.20type.3F/near/218125195).

This uses a command line option instead of a feature gate so it's easier to pass it to all crates at once. Theoretically it's better to use a feature gate instead so that it's easier for people to use on docs.rs, but I'm also not terribly concerned with how easy it to use a temporary hack.

Addresses https://github.com/rust-lang/rust/issues/77459.
2020-12-01 10:50:18 +00:00
Mara Bos
36ce8db229
Rollup merge of #79522 - ehuss:lint-check-validate, r=Mark-Simulacrum
Validate lint docs separately.

This addresses some concerns raised in https://github.com/rust-lang/rust/pull/76549#issuecomment-727638552 about errors with the lint docs being confusing and cumbersome. Errors from validating the lint documentation were being generated during `x.py doc` (and `x.py dist`), since extraction and validation are being done in a single step. This changes it so that extraction and validation are separated, so that `x.py doc` will not error if there is a validation problem, and tests are moved to `x.py test src/tools/lint-docs`.

This includes the following changes:

* Separate validation to `x.py test`.
* Added some more documentation on how to more easily modify and test the docs.
* Added more help to the error messages to hopefully provide more information on how to fix things.

The first commit just moves the code around, so you may consider looking at the other commits for a smaller diff.
2020-12-01 10:50:16 +00:00
Joshua Nelson
95a6427d2c Add -Z normalize-docs and enable it for compiler docs 2020-11-29 17:21:24 -05:00
Joshua Nelson
872acb0c35 Move src/test/rustdoc-ui intra-doc tests into a subdirectory
This also changes the builder to allow using
`x.py test src/test/rustdoc-ui/intra-doc`; before, it would panic that
no paths were found.
2020-11-28 21:14:09 -05:00
nooberfsh
331d52643a Fix a bootstrap comment 2020-11-29 10:02:24 +08:00
Eric Huss
d2d91b42a5 lint-docs: Add --validate flag to validate lint docs separately. 2020-11-28 13:39:02 -08:00
Joseph Rafael Ferrer
6b47920c69 Convert UNC path to local path to satisfy install script on Windows 2020-11-23 22:37:51 +08:00
bors
d806d65657 Auto merge of #79115 - cuviper:rust-description, r=Mark-Simulacrum
x.py: allow a custom string appended to the version

This adds `rust.description` to the config as a descriptive string to be
appended to `rustc --version` output, which is also used in places like
debuginfo `DW_AT_producer`. This may be useful for supplementary build
information, like distro-specific package versions.

For example, in Fedora 33, `gcc --version` outputs:

    gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6)

With this change, we can add similar vendor info to `rustc --version`.
2020-11-21 15:43:09 +00:00
Dylan DPC
d5ee4edee1
Rollup merge of #79183 - cuviper:compiletest-test-sysroot, r=Mark-Simulacrum
Make compiletest testing use the local sysroot

We already set `compiletest` to use the local sysroot in #68019, but
that missed the configuration for testing `compiletest` itself.
2020-11-19 23:58:40 +01:00
Dylan DPC
de62272548
Rollup merge of #79156 - jyn514:relative-llvm, r=Mark-Simulacrum
Allow using `download-ci-llvm` from directories other than the root

Previously, bootstrap.py would attempt to find the LLVM commit from
`src/llvm-project`. However, it assumed it was always being run from the
top-level directory, which isn't always the case.

Before:

```
downloading https://ci-artifacts.rust-lang.org/rustc-builds//rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz

curl: (22) The requested URL returned error: 404
failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmppyh4w8 https://ci-artifacts.rust-lang.org/rustc-builds//rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz
Build completed unsuccessfully in 0:00:02
```

After:

```
downloading https://ci-artifacts.rust-lang.org/rustc-builds/430feb24a46993e5073c1bb1b39da190d83fa2bf/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz
 ###################################################################################################################################################################### 100.0%
extracting /home/joshua/rustc/src/bootstrap/build/cache/llvm-430feb24a46993e5073c1bb1b39da190d83fa2bf-False/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz
```

r? ```@Mark-Simulacrum```
cc ```@pnkfelix```
2020-11-19 16:26:42 +01:00
Dylan DPC
3b857a5a3a
Rollup merge of #79155 - 12101111:fix-profiler-config, r=Mark-Simulacrum
fix handling the default config for profiler and sanitizers

#78354 don't handle the case that user don't add any target-specific config in `[target.*]` of `config.toml`:
```toml
changelog-seen = 2
[llvm]
link-shared = true
[build]
sanitizers = true
profiler = true
[install]
[rust]
[dist]
```

The previes code handle the default config in `Config::prase()`:

```rust
target.sanitizers = cfg.sanitizers.unwrap_or(build.sanitizers.unwrap_or_default());
target.profiler = cfg.profiler.unwrap_or(build.profiler.unwrap_or_default());
config.target_config.insert(TargetSelection::from_user(&triple), target);
```

In this case, `toml.target` don't contain any target, so the above code won't execute. Instead, a default `Target` is insert in c919f490bb/src/bootstrap/sanity.rs (L162-L166)

The default value for `bool` is false, hence the issue in #79124

This fix change the type of `sanitizers` and `profiler` to `Option<bool>`, so the default value is `None`, and fallback config is handled in  `Config::sanitizers_enabled` and `Config::profiler_enabled`

fix #79124

cc `@Mark-Simulacrum` `@richkadel`
2020-11-19 16:26:40 +01:00
Josh Stone
566e8771a5 Make compiletest testing use the local sysroot
We already set `compiletest` to use the local sysroot in #68019, but
that missed the configuration for testing `compiletest` itself.
2020-11-18 14:41:27 -08:00
Joshua Nelson
5163912e56 Allow using download-ci-llvm from directories other than the root
Previously, bootstrap.py would attempt to find the LLVM commit from
`src/llvm-project`. However, it assumed it was always being run from the
top-level directory, which isn't always the case.

Before:

```
downloading https://ci-artifacts.rust-lang.org/rustc-builds//rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz

curl: (22) The requested URL returned error: 404
failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmppyh4w8 https://ci-artifacts.rust-lang.org/rustc-builds//rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz
Build completed unsuccessfully in 0:00:02
```

After:

```
downloading https://ci-artifacts.rust-lang.org/rustc-builds/430feb24a46993e5073c1bb1b39da190d83fa2bf/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz
 ###################################################################################################################################################################### 100.0%
extracting /home/joshua/rustc/src/bootstrap/build/cache/llvm-430feb24a46993e5073c1bb1b39da190d83fa2bf-False/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.gz
```
2020-11-17 22:09:08 -05:00
12101111
36972b0d63 fix handling the default config for profiler and sanitizers 2020-11-18 10:33:07 +08:00
Pietro Albini
d17874ff60 bootstrap: use the same version number for rustc and cargo
Historically the stable tarballs were named after the version number of
the specific tool, instead of the version number of Rust. For example,
both of the following tarballs were part of the same release:

    rustc-1.48.0-x86_64-unknown-linux-gnu.tar.xz
    cargo-0.49.0-x86_64-unknown-linux-gnu.tar.xz

PR #77336 changed the dist code to instead use Rust's version number for
all the tarballs, regardless of the tool they contain:

    rustc-1.48.0-x86_64-unknown-linux-gnu.tar.xz
    cargo-1.48.0-x86_64-unknown-linux-gnu.tar.xz

Because of that there is no need anymore to have a separate `cargo`
field in src/stage0.txt, as the Cargo version will always be the same as
the rustc version. This PR removes the field, simplifying the code and
the maintenance work required while producing releases.
2020-11-17 09:14:02 -05:00
Josh Stone
5f08568042 x.py: allow a custom string appended to the version
This adds `rust.description` to the config as a descriptive string to be
appended to `rustc --version` output, which is also used in places like
debuginfo `DW_AT_producer`. This may be useful for supplementary build
information, like distro-specific package versions.

For example, in Fedora 33, `gcc --version` outputs:

    gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6)

With this change, we can add similar vendor info to `rustc --version`.
2020-11-16 14:08:21 -08:00
bors
f4d014cee7 Auto merge of #79074 - Mark-Simulacrum:fix-ci-llvm, r=jyn514
Install CI llvm into the library directory

In other words, my concern in https://github.com/rust-lang/rust/issues/78932#issuecomment-725781767 was perfectly justified by something we were already doing. For now just special case CI LLVM, but in the future we may want a more general fix.

Fixes #79071.

r? `@alexcrichton`
2020-11-16 00:40:04 +00:00
Mark Rousskov
4feaa35f39 Install CI llvm into the library directory 2020-11-15 08:59:53 -05:00
Jonas Schievink
c8ac745040
Rollup merge of #79059 - jyn514:cranelift, r=Mark-Simulacrum
Print 'checking cranelift artifacts' to easily separate it from other artifacts

Before:

```
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking rustdoc v0.0.0 (/home/joshua/rustc/src/librustdoc)
    Checking rustdoc-tool v0.0.0 (/home/joshua/rustc/src/tools/rustdoc)
    Finished release [optimized] target(s) in 2.08s
    Checking regalloc v0.0.31
    Checking gimli v0.22.0
    Checking object v0.21.1
    Checking cranelift-codegen v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-module v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-native v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-frontend v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-object v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-simplejit v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking rustc_codegen_cranelift v0.1.0 (/home/joshua/rustc/compiler/rustc_codegen_cranelift)
    Finished release [optimized] target(s) in 10.55s

```

After:

```
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking rustdoc v0.0.0 (/home/joshua/rustc/src/librustdoc)
    Checking rustdoc-tool v0.0.0 (/home/joshua/rustc/src/tools/rustdoc)
    Finished release [optimized] target(s) in 2.08s
Checking cranelift artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking cranelift-codegen v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
```

r? `@bjorn3`
`@bors` delegate=bjorn3
2020-11-15 13:40:05 +01:00
Jonas Schievink
e0c378a673
Rollup merge of #79004 - jyn514:bacon, r=Mark-Simulacrum
Add `--color` support to bootstrap

When running under external utilities which wrap x.py, it can be convenient to force color support on.
2020-11-15 13:39:45 +01:00
Dylan DPC
e3d52b8051
Rollup merge of #78986 - Mark-Simulacrum:fix-llvm, r=alexcrichton
Avoid installing external LLVM dylibs

If the LLVM was externally provided, then we don't currently copy artifacts into
the sysroot. This is not necessarily the right choice (in particular, it will
require the LLVM dylib to be in the linker's load path at runtime), but the
common use case for external LLVMs is distribution provided LLVMs, and in that
case they're usually in the standard search path (e.g., /usr/lib) and copying
them here is going to cause problems as we may end up with the wrong files and
isn't what distributions want.

This behavior may be revisited in the future though.

Fixes #78932.
2020-11-15 03:02:55 +01:00
Dylan DPC
3fe2abafe5
Rollup merge of #78968 - zec:add-llvm-as, r=Mark-Simulacrum
Include llvm-as in llvm-tools-preview component

Including `llvm-as` adds the ability to include assembly language fragments that can be inlined using LTO while making sure the correct version of LLVM is always used.
2020-11-15 03:02:49 +01:00
Joshua Nelson
6900dcbc03 Print 'checking cranelift artifacts' to easily separate it from other artifacts
Before:

```
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking rustdoc v0.0.0 (/home/joshua/rustc/src/librustdoc)
    Checking rustdoc-tool v0.0.0 (/home/joshua/rustc/src/tools/rustdoc)
    Finished release [optimized] target(s) in 2.08s
    Checking regalloc v0.0.31
    Checking gimli v0.22.0
    Checking object v0.21.1
    Checking cranelift-codegen v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-module v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-native v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-frontend v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-object v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking cranelift-simplejit v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
    Checking rustc_codegen_cranelift v0.1.0 (/home/joshua/rustc/compiler/rustc_codegen_cranelift)
    Finished release [optimized] target(s) in 10.55s

```

After:

```
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking rustdoc v0.0.0 (/home/joshua/rustc/src/librustdoc)
    Checking rustdoc-tool v0.0.0 (/home/joshua/rustc/src/tools/rustdoc)
    Finished release [optimized] target(s) in 2.08s
Checking cranelift artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking cranelift-codegen v0.67.0 (https://github.com/bytecodealliance/wasmtime/?branch=main#44cbdece)
```
2020-11-14 17:37:37 -05:00
Joshua Nelson
31741aad39 Add --color support to bootstrap
This allows using bootstrap with https://github.com/Canop/bacon.
2020-11-12 16:31:21 -05:00
Mark Rousskov
3747df71fa Avoid installing external LLVM dylibs
If the LLVM was externally provided, then we don't currently copy artifacts into
the sysroot. This is not necessarily the right choice (in particular, it will
require the LLVM dylib to be in the linker's load path at runtime), but the
common use case for external LLVMs is distribution provided LLVMs, and in that
case they're usually in the standard search path (e.g., /usr/lib) and copying
them here is going to cause problems as we may end up with the wrong files and
isn't what distributions want.

This behavior may be revisited in the future though.
2020-11-12 09:49:45 -05:00
DevJPM
6830f1c6e2 Bump the minimal supported LLVM version in the bootstrapping code to 9.0 2020-11-12 14:39:47 +01:00
Zachary Catlin
e4a43fce3b Merge changes from rust-lang/rust 2020-11-11 21:17:45 -05:00
Zachary Catlin
562d50eb7b Include llvm-as in llvm-tools-preview component
Including llvm-as adds the ability to include assembly language fragments
that can be inlined using LTO.
2020-11-11 21:10:31 -05:00
Jonas Schievink
61c0a2c4ac
Rollup merge of #78947 - dalance:llvm_cov, r=Mark-Simulacrum
Ship llvm-cov through llvm-tools

`llvm-cov` is used to generate coverage report with LLVM InstrProf-based code coverage #34701.
So if `llvm-cov` is shipped through llvm-tools, users can try it easily accorging to the instruction of [The Rust Unstable Book](https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/source-based-code-coverage.html).
2020-11-11 20:59:14 +01:00
Jonas Schievink
194b96852f
Rollup merge of #78354 - 12101111:rustbuild_profiler, r=Mark-Simulacrum
Support enable/disable sanitizers/profiler per target

This PR add options under `[target.*]` of `config.toml` which can enable or disable sanitizers/profiler runtime for corresponding target.
If these options are empty, the global options under `[build]` will take effect.

Fix #78329
2020-11-11 20:58:56 +01:00
bors
7afc517230 Auto merge of #78790 - Gankra:rust-src-vendor, r=Mark-Simulacrum
Vendor libtest's dependencies in the rust-src component

This is the Rust side of https://github.com/rust-lang/wg-cargo-std-aware/issues/23

Note that this won't produce a useful result for `cargo -Zbuild-std` if there are multiple versions of a crate vendored, but will otherwise produce a valid vendor dir.

See https://github.com/rust-lang/cargo/pull/8834 for the other half of this change.
2020-11-11 16:24:43 +00:00
dalance
2453ce717a Ship llvm-cov through llvm-tools 2020-11-12 00:03:09 +09:00