A new option is introduced under the `[llvm]` section of `config.toml`,
`targets`, for overriding the list of LLVM targets to build support for.
The option is passed through to LLVM configure script. Also notes are
added about the implications of (ab)using the option; since the default
is not changed, and users of the option are expected to know what
they're doing anyway (as every porter should), the impact should be
minimal.
Fixes#38200.
rustbuild: Hotfix to unbreak nightly
Fixes an oversight unnoticed in #38468 that eventually broke nightly packaging. I didn't realize this until some moments ago, when I finally found out the failure is actually deterministic. Many apologies for eating 3 nightlies during the holidays.
r? @alexcrichton
This commit primarily starts supporting the `DESTDIR` environment variable like
the old build system. Along the way this brings `config.toml` up to date with
support in `config.mk` with install options supported.
Closes#38441
Teach `rustdoc --test` about `--sysroot`, pass it when testing rust
This permits rustdoc tests to work in stage0.
Logical continuation of #36586.
Snippet from https://github.com/rust-lang/rust/issues/38575#issuecomment-269090724:
> it should actually be possible to run all the libstd tests immediately after creating std of stage0-out - there's no reason to build librustc at all if you've just made a change to (for example) libcollections, `./x.py test src/libcollections --stage 0 -v --incremental` should just work
This PR makes it so (or appears to in my testing).
r? @alexcrichton
`arr` is the actual list of targets participating in steps construction,
but due to #38468 the hosts array now consists of only the build triple
for the `dist` steps, hence all non-build-triple targets are lost for
the host-only rules.
Fix this by using the original non-shadowed hosts array in `arr`
calculation. This should unbreak the nightly packaging process.
Fixes#38637.
The comment touched, as originally written, only concerned itself with
the `test` steps. However, since #38468 the `arr` variable actually has
gained an indirect relationship with the `dist` steps too. The comment
failed to convey the extra meaning, contributing to the misunderstanding
which eventually lead to #38637. Fix that by moving the comment into the
right place near the relevant condition, and properly documenting
`arr`'s purpose.
- `--emit=asm --target=nvptx64-nvidia-cuda` can be used to turn a crate
into a PTX module (a `.s` file).
- intrinsics like `__syncthreads` and `blockIdx.x` are exposed as
`"platform-intrinsics"`.
- "cabi" has been implemented for the nvptx and nvptx64 architectures.
i.e. `extern "C"` works.
- a new ABI, `"ptx-kernel"`. That can be used to generate "global"
functions. Example: `extern "ptx-kernel" fn kernel() { .. }`. All
other functions are "device" functions.
initial SPARC support
### UPDATE
Can now compile `no_std` executables with:
```
$ cargo new --bin app && cd $_
$ edit Cargo.toml && tail -n2 $_
[dependencies]
core = { path = "/path/to/rust/src/libcore" }
$ edit src/main.rs && cat $_
#![feature(lang_items)]
#![no_std]
#![no_main]
#[no_mangle]
pub fn _start() -> ! {
loop {}
}
#[lang = "panic_fmt"]
fn panic_fmt() -> ! {
loop {}
}
$ edit sparc-none-elf.json && cat $_
{
"arch": "sparc",
"data-layout": "E-m:e-p:32:32-i64:64-f128:64-n32-S64",
"executables": true,
"llvm-target": "sparc",
"os": "none",
"panic-strategy": "abort",
"target-endian": "big",
"target-pointer-width": "32"
}
$ cargo rustc --target sparc-none-elf -- -C linker=sparc-unknown-elf-gcc -C link-args=-nostartfiles
$ file target/sparc-none-elf/debug/app
app: ELF 32-bit MSB executable, SPARC, version 1 (SYSV), statically linked, not stripped
$ sparc-unknown-elf-readelf -h target/sparc-none-elf/debug/app
ELF Header:
Magic: 7f 45 4c 46 01 02 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, big endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Sparc
Version: 0x1
Entry point address: 0x10074
Start of program headers: 52 (bytes into file)
Start of section headers: 1188 (bytes into file)
Flags: 0x0
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 2
Size of section headers: 40 (bytes)
Number of section headers: 14
Section header string table index: 11
$ sparc-unknown-elf-objdump -Cd target/sparc-none-elf/debug/app
target/sparc-none-elf/debug/app: file format elf32-sparc
Disassembly of section .text:
00010074 <_start>:
10074: 9d e3 bf 98 save %sp, -104, %sp
10078: 10 80 00 02 b 10080 <_start+0xc>
1007c: 01 00 00 00 nop
10080: 10 80 00 02 b 10088 <_start+0x14>
10084: 01 00 00 00 nop
10088: 10 80 00 00 b 10088 <_start+0x14>
1008c: 01 00 00 00 nop
```
---
Someone wants to attempt launching some Rust [into space](https://www.reddit.com/r/rust/comments/5h76oa/c_interop/) but their platform is based on the SPARCv8 architecture. Let's not block them by enabling LLVM's SPARC backend.
Something very important that they'll also need is the "cabi" stuff as they'll be embedding some Rust code into a bigger C application (i.e. heavy use of `extern "C"`). The question there is what name(s) should we use for "target_arch" as the "cabi" implementation [varies according to that parameter](https://github.com/rust-lang/rust/blob/1.13.0/src/librustc_trans/abi.rs#L498-L523).
AFAICT, SPARCv8 is a 32-bit architecture and SPARCv9 is a 64-bit architecture. And, LLVM uses `sparc`, `sparcv9` and `sparcel` for [the architecture triple](ac1c94226e/include/llvm/ADT/Triple.h (L67-L69)) so perhaps we should use `target_arch = "sparc"` (32-bit) and `target_arch = "sparcv9"` (64-bit) as well.
r? @alexcrichton This PR only enables this LLVM backend when rustbuild is used. Do I also need to implement this for the old Makefile-based build system? Or are all our nightlies now being generated using rustbuild?
cc @brson
rustbuild: Actually test musl on the musl bot
A typo unfortunately meant that we haven't been testing musl for a bit, so now
it's time to discover if we accidentally introduced a regression!
rustbuild: Run debuginfo tests by default
This fixes an accidental regression in rustbuild which stopped running debuginfo
tests by default. Here we flag the test suites as `default(true)` to ensure that
they're run on bots, for example.
rustbuild: Update Cargo download location
I updated the beta compiler used to bootstrap the master branch in #38438 with
the intention of fixing Travis OSX linkage issues but I mistakenly forgot that
the PR only updated rustc, not Cargo itself. Cargo has a new release process
with downloads in a different location, so this commit updates rustbuild to
download from this new location by tracking revisions instead of Cargo nightly
dates.
rustbuild: Eliminate duplication of dist tarballs
Fixes#38365 by not constructing the duplicate steps in the first place, as suggested. The source package step is lacking the check as in other steps, so it is added as well.
Tested locally with the `alexcrichton/rust-slave-linux-cross:2016-11-11` container (with the build slave init replaced with no-op, of course).
r? @alexcrichton
adaptation to rustbuild for openbsd
Since the switch to rustbuild, the build for openbsd is broken:
- [X] `ar` inference based on compiler name is wrong (OpenBSD usually use `egcc`, but `ear` doesn't exist)
- [X] `make` isn't GNU-make under OpenBSD (and others BSD platforms)
- [x] `stdc++` isn't the right stdc++ library to link with (it should be `estdc++`)
- [x] corrects tests that don't pass anymore (problems related to rustbuild)
r? @alexcrichton
rustbuild: Stop building docs for libtest by default
They cause the search index from the std docs to get overwritten just like #34800.
Part of #38319.
Add prefix to config.toml
This allows `rustbuild` to be used to install to a prefix.
```toml
[build]
prefix = "/path/to/install"
```
For example, the following `config.toml` will cause `x.py dist --install` to install to `/path/to/install`
This fixes an accidental regression in rustbuild which stopped running debuginfo
tests by default. Here we flag the test suites as `default(true)` to ensure that
they're run on bots, for example.
We only want to package each host/target once for `dist`. The obvious
solution takes the form of step dependency, which is implemented at
least for the `dist-rustc` step. Unfortunately since the steps are
created from `hosts x targets` during planning and *not* de-duplicated
afterwards, the problem still persists.
We therefore move the check inside `plan()` instead, to avoid creating
the duplicate steps in the first place.
I updated the beta compiler used to bootstrap the master branch in #38438 with
the intention of fixing Travis OSX linkage issues but I mistakenly forgot that
the PR only updated rustc, not Cargo itself. Cargo has a new release process
with downloads in a different location, so this commit updates rustbuild to
download from this new location by tracking revisions instead of Cargo nightly
dates.
add preliminary support for incremental compilation to rustbuild.py
This implements the integration described in #37929. It requires the use of a local nightly as your bootstrap compiler. The setup is described in `src/bootstrap/README.md`.
This does NOT implement the "copy stage0 libs to stage1" optimization described in #37929, just because that seems orthogonal to me.
In local testing, I do not yet see any incremental re-use when building rustc. I'm not sure why that is, more investigation needed.
(For these reasons, this is not marked as fixing the relevant issue.)
r? @alexcrichton -- I included one random cleanup (`Step::noop()`) that turned out to not be especially relevant. Feel free to tell me you liked it better the old way.
rustbuild: Fix `copy` helper with existing files
This erroneously truncated files when the destination already existed and was an
existing hard link to the source. This in turn caused weird bugs!
Closes#37745
This not only avoids the small – and unnecessary – constant overhead for each compiler invocation,
but also helps somewhat by only having “correct” rustc processes to look for in `/proc/`.
This also makes the wrapper behave effectively as a regular exec wrapper its intended to be.
rustbuild: Fix LC_ID_DYLIB directives on OSX
Currently libraries installed by rustbuild on OSX have an incorrect
`LC_ID_DYLIB` directive located in the dynamic libraries that are
installed. The directive we expect looks like:
@rpath/libstd.dylib
Which means that if you want to find that dynamic library you should
look at the dylib's other `@rpath` directives. Typically our `@rpath`
directives look like `@loader_path/../lib` for the compiler as that's
where the installed libraries will be located. Currently, though,
rustbuild produces dylibs with the directive that looks like:
/Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps/libstd-713ad88203512705.dylib
In other words, the build directory is encoded erroneously. The compiler
already [knows how] to change this directive, but it only passes that
argument when `-C rpath` is also passed. The rustbuild system, however,
explicitly [does not pass] this option explicitly and instead bakes its
own. This logic then also erroneously didn't pass `-Wl,-install_name`
like the compiler.
[knows how]: 4a008cccaa/src/librustc_trans/back/linker.rs (L210-L214)
[does not pass]: 4a008cccaa/src/bootstrap/bin/rustc.rs (L133-L158)
To fix this regression this patch introduces a new `-Z` flag, `-Z
osx-rpath-install-name` which basically just forces the compiler to take
the previous `-install_name` branch when creating a dynamic library.
Hopefully we can sort out a better rpath story in the future, but for
now this "hack" should suffice in getting our nightly builds back to the
same state as before.
Closes#38430
rustbuild: Create directories in mingw dist
Previously we accidentally relied on the mingw dist step running last, but the
step just needed to ensure the directories were created.
Currently libraries installed by rustbuild on OSX have an incorrect
`LC_ID_DYLIB` directive located in the dynamic libraries that are
installed. The directive we expect looks like:
@rpath/libstd.dylib
Which means that if you want to find that dynamic library you should
look at the dylib's other `@rpath` directives. Typically our `@rpath`
directives look like `@loader_path/../lib` for the compiler as that's
where the installed libraries will be located. Currently, though,
rustbuild produces dylibs with the directive that looks like:
/Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps/libstd-713ad88203512705.dylib
In other words, the build directory is encoded erroneously. The compiler
already [knows how] to change this directive, but it only passes that
argument when `-C rpath` is also passed. The rustbuild system, however,
explicitly [does not pass] this option explicitly and instead bakes its
own. This logic then also erroneously didn't pass `-Wl,-install_name`
like the compiler.
[knows how]: 4a008cccaa/src/librustc_trans/back/linker.rs (L210-L214)
[does not pass]: 4a008cccaa/src/bootstrap/bin/rustc.rs (L133-L158)
To fix this regression this patch introduces a new `-Z` flag, `-Z
osx-rpath-install-name` which basically just forces the compiler to take
the previous `-install_name` branch when creating a dynamic library.
Hopefully we can sort out a better rpath story in the future, but for
now this "hack" should suffice in getting our nightly builds back to the
same state as before.
Closes#38430
rustbuild: Add sccache support
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.
The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
---
Note that this does not currently change Windows [due to previously written up issues](https://github.com/rust-lang/rust/issues/38119#issuecomment-266631585). Despite that, however, I was curious to get timings for the builds on Travis to see what ranges we're working with. As a result, this is a WIP PR I'm using to gauge build times and such.
This erroneously truncated files when the destination already existed and was an
existing hard link to the source. This in turn caused weird bugs!
Closes#37745
rustbuild: Package rust-mingw by default
This fixes the `make dist` step on MinGW to package the `rust-mingw` component
by default. This should hopefully be the last step in fixing nightlies.
rustbuild: Add cli option --keep-stage
This option is intended to be used like:
./x.py build --stage 1 --keep-stage 0
Which skips all stage 0 steps, so that stage 1 can be recompiled
directly (even if for example libcore has changes).
This is useful when working on `cfg(not(stage0))` parts of the
libraries or when re-running stage 1 tests in libraries in general.
Fixes#38326
