LLVM 4.0 Upgrade
Since nobody has done this yet, I decided to get things started:
**Todo:**
* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [x] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image
---
Closes#37609.
---
**Test results:**
Everything is green 🎉
This commit enables the `rust-analysis` package to be produced for all targets
that are part of the `dist-*` suite of docker images on Travis. Currently
these packages are showing up with `available = false` in the
`channel-rust-nightly.toml` manifest where we'd prefer to have them show up for
all targets.
Unfortunately rustup isn't handling the `available = false` section well right
now, so this should also inadvertently fix the nightly regression.
travis: Use upstream LLVM repositories for Fuchsia
The Fuchsia copies of LLVM repositories contain additional patches
for work-in-progress features and there is some amount of churn that
may break Rust. Use upstream LLVM repositories instead for building
the toolchain used by the Fuchsia builder.
dist-powerpc-linux: use a pure 32-bit CPU profile
With `-mcpu=power4`, code might use instructions like `fcfid`, excluding
older CPUs like the PowerPC G4, which apparently some users would like
to use. The generic `-mcpu=powerpc` should stick to pure 32-bit PowerPC
instructions.
Fixesrust-lang/cargo#3852.
The Fuchsia copies of LLVM repositories contain additional patches
for work-in-progress features and there is some amount of churn that
may break Rust. Use upstream LLVM repositories instead for building
the toolchain used by the Fuchsia builder.
With `-mcpu=power4`, code might use instructions like `fcfid`, excluding
older CPUs like the PowerPC G4, which apparently some users would like
to use. The generic `-mcpu=powerpc` should stick to pure 32-bit PowerPC
instructions.
Fixesrust-lang/cargo#3852.
I've tracked down what I believe is the last spurious sccache failure on #40240
to behavior in mio (carllerche/mio#583), and this commit updates the binaries to
a version which has that fix incorporated.
Previously we would use one builder on Travis to produce two sets of host
compilers for two different targets. Unfortunately though we've recently
increased how much we're building for each target so this is starting to take
unnecessarily long (#40804). This commit splits the dist builders in two by
ensuring that we only dist one target on each builder, which should take a much
shorter amount of time. This should also unblock other work such as landing the
RLS (#40584).
Attempt to cache git modules
Partial resolution of #40772, appveyor remains to be done once travis looks like it's working ok.
The approach in this PR is based on the `--reference` flag to `git-clone`/`git-submodule --update` and is a compromise based on the current limitations of the tools we're using.
The ideal would be:
1. have a cached pristine copy of rust-lang/rust master in `$HOME/rustsrc` with all submodules initialised
2. clone the PR branch with `git clone --recurse-submodules --reference $HOME/rustsrc git@github.com:rust-lang/rust.git`
This would (in the nonexistent ideal world) use the pristine copy as an object cache for the top level repo and all submodules, transferring over the network only the changes on the branch. Unfortunately, a) there is no way to manually control the initial clone with travis and b) even if there was, cloned submodules don't use the submodules of the reference as an object cache. So the steps we end up with are:
1. have a cached pristine copy of rust-lang/rust master in `$HOME/rustsrc` with all submodules initialised
2. have a cloned PR branch
3. extract the path of each submodule, and explicitly `git submodule update --init --reference $HOME/rustsrc/$module $module` (i.e. point directly to the location of the pristine submodule repo) for each one
I've also taken some care to make this forward compatible, both for adding and removing submodules.
r? @alexcrichton
Let's have this layer be towards the end as we're emprically changing sccache
more than we're changing the rest of the image, so this'll allow us to reuse as
much of the cached image as possible.
Use the "official" cross compiler for NetBSD
The current NetBSD cross compiler is lacking, for example `std::thread` is not available (which causes problems for LLVM 4.0). This PR uses the official netbsd build system to compiler the cross compiler.
@alexcrichton: Can you please mirror `ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-7.0/source/sets/{src,gnusrc,sharesrc,syssrc}.tgz`. (Optionally you may want to use NetBSD versions 7.0.2 or 7.1, in that case you'll probably want to update the binary downloads used today as well).
I'll update the URL's afterwards (or feel free to use "allow edits from maintainers").
r? @alexcrichton
travis: Ensure cargo links libcurl statically
We don't want a dynamic dependency in the library that we ship, so link it
statically by configuring curl-sys's build script to not pick up the system
version via pkg-config.
We don't want a dynamic dependency in the library that we ship, so link it
statically by configuring curl-sys's build script to not pick up the system
version via pkg-config.
rustbuild: Build documentation for `proc_macro`
This commit fixes#38749 by building documentation for the `proc_macro` crate by
default for configured hosts. Unfortunately did not turn out to be a trivial
fix. Currently rustbuild generates documentation into multiple locations: one
for std, one for test, and one for rustc. The initial fix for this issue simply
actually executed `cargo doc -p proc_macro` which was otherwise completely
elided before.
Unfortunately rustbuild was the left to merge two documentation trees together.
One for the standard library and one for the rustc tree (which only had docs for
the `proc_macro` crate). Rustdoc itself knows how to merge documentation files
(specifically around search indexes, etc) but rustbuild was unaware of this, so
an initial fix ended up destroying the sidebar and the search bar from the
libstd docs.
To solve this issue the method of documentation has been tweaked slightly in
rustbuild. The build system will not use symlinks (or directory junctions on
Windows) to generate all documentation into the same location initially. This'll
rely on rustdoc's logic to weave together all the output and ensure that it ends
up all consistent.
Closes#38749
travis: Attempt to debug sccache failures
I can't find anything that'd cause unexpected EOF in the source, so let's try
taking a look at the error logs on failures.
This commit fixes#38749 by building documentation for the `proc_macro` crate by
default for configured hosts. Unfortunately did not turn out to be a trivial
fix. Currently rustbuild generates documentation into multiple locations: one
for std, one for test, and one for rustc. The initial fix for this issue simply
actually executed `cargo doc -p proc_macro` which was otherwise completely
elided before.
Unfortunately rustbuild was the left to merge two documentation trees together.
One for the standard library and one for the rustc tree (which only had docs for
the `proc_macro` crate). Rustdoc itself knows how to merge documentation files
(specifically around search indexes, etc) but rustbuild was unaware of this, so
an initial fix ended up destroying the sidebar and the search bar from the
libstd docs.
To solve this issue the method of documentation has been tweaked slightly in
rustbuild. The build system will not use symlinks (or directory junctions on
Windows) to generate all documentation into the same location initially. This'll
rely on rustdoc's logic to weave together all the output and ensure that it ends
up all consistent.
Closes#38749
This change introduces a Dockerfile and script which builds a complete
Fuchsia toolchain which can be used to build Rust distribution for
Fuchsia. We only support cross-compiling at the moment, hence only
setting the target.
Travis only gives us 30GB disk space and we don't currently have an option to
increase that. Each musl target generates "hello world" binaries of about 3.5MB
in size, and we're testing two targets in the same image. We have around 3k
run-pass tests and 2 musl targets which works out to around 20GB. That's
dangerously close to the limit and is causing PRs to bounce.
This PR splits up the builder in two, one for x86_64 musl and the other for
i686. Hopefully that'll keep us under the disk limit.
Closes#40359
PRs can't land againt beta right now because the android bot is filling up on
disk space. I don't really know what's going on but the android bot is the
longest one to run anyway so it'll benefit from being split up regardless.
