rustbuild: Build documentation for `proc_macro`
This commit fixes#38749 by building documentation for the `proc_macro` crate by
default for configured hosts. Unfortunately did not turn out to be a trivial
fix. Currently rustbuild generates documentation into multiple locations: one
for std, one for test, and one for rustc. The initial fix for this issue simply
actually executed `cargo doc -p proc_macro` which was otherwise completely
elided before.
Unfortunately rustbuild was the left to merge two documentation trees together.
One for the standard library and one for the rustc tree (which only had docs for
the `proc_macro` crate). Rustdoc itself knows how to merge documentation files
(specifically around search indexes, etc) but rustbuild was unaware of this, so
an initial fix ended up destroying the sidebar and the search bar from the
libstd docs.
To solve this issue the method of documentation has been tweaked slightly in
rustbuild. The build system will not use symlinks (or directory junctions on
Windows) to generate all documentation into the same location initially. This'll
rely on rustdoc's logic to weave together all the output and ensure that it ends
up all consistent.
Closes#38749
This commit fixes#38749 by building documentation for the `proc_macro` crate by
default for configured hosts. Unfortunately did not turn out to be a trivial
fix. Currently rustbuild generates documentation into multiple locations: one
for std, one for test, and one for rustc. The initial fix for this issue simply
actually executed `cargo doc -p proc_macro` which was otherwise completely
elided before.
Unfortunately rustbuild was the left to merge two documentation trees together.
One for the standard library and one for the rustc tree (which only had docs for
the `proc_macro` crate). Rustdoc itself knows how to merge documentation files
(specifically around search indexes, etc) but rustbuild was unaware of this, so
an initial fix ended up destroying the sidebar and the search bar from the
libstd docs.
To solve this issue the method of documentation has been tweaked slightly in
rustbuild. The build system will not use symlinks (or directory junctions on
Windows) to generate all documentation into the same location initially. This'll
rely on rustdoc's logic to weave together all the output and ensure that it ends
up all consistent.
Closes#38749
This commit attempts to move more network operations to being retryable through
various operations. For example git submodule updates, downloading snapshots,
etc, are now all in retryable steps.
Hopefully this commit can cut down on the number of network failures we've been
seeing!
This commit adds three new builders, one OSX, one Linux, and one MSVC, which
will produce "nightlies" with LLVM assertions disabled. Currently all nightly
releases have LLVM assertions enabled to catch bugs before they reach the
beta/stable channels. The beta/stable channels, however, do not have LLVM
assertions enabled.
Unfortunately though projects like Servo are stuck on nightlies for the near
future at least and are also suffering very long compile times. The purpose of
this commit is to provide artifacts to these projects which are not distributed
through normal channels (e.g. rustup) but are provided for developers to use
locally if need be.
Logistically these builds will all be uploaded to `rustc-builds-alt` instead of
the `rustc-builds` folder of the `rust-lang-ci` bucket. These builds will stay
there forever (until cleaned out if necessary) and there are no plans to
integrate this with rustup and/or the official release process.
This commit updates the compilers for many of the artifacts that we're producing
on Travis. These compilers are all compiled by crosstool-ng as they're currently
done for the images in which we're building all our cross compiled compilers.
The purpose of this commit is that when we ship binaries the artifacts won't
require a newer glibc, but rather be as compatible as possible with Linux
distributions by working with a very old version of glibc.
This commit always allocates a new matrix entry for the i686/x86_64 builder.
This builder is dedicated to just producing artifacts and eventually we'll
expand it to building other tools like Cargo and the RLS. The other builders
testing i686 and x86_64 won't use these historical toolchains.
This commit passes the `--release-channel=nightly` flag to all images which have
the `DEPLOY` flag set. This means that we'll name artifacts and the compiler
appropriately.
This reworks a bit how arguments are passed, but for now doesn't change what's
already being passed. Eventually we'll want to avoid enabling debug assertions
and llvm assertions for *all* releases, but I figure we can tackle that a little
bit more down the road.
travis: Expand dist builder coverage
This commit adds six new travis matrix entires for doing cross-compiled
distribution builds of the compiler. The support added in #38731 allows us to
quickly compile a complete suite of distribution artifacts for cross-compiled
platforms, and currently each matrix entry (when fully cached) clocks in around
an hour to finish. Note that a full test run typically takes about two hours
right now.
With further optimizations coming down the pike in #39026 this commit also
starts doubling up cross-compiled distribution builders on each matrix entry. We
initially planned to do one build per entry, but it's looking like we may be
able to get by with more than one in each entry. Depending on how long these
builds take we may even be able to up it to three, but we'll start with two
first.
This commit then completes the suite of cross-compiled compilers that we're
going to compile, adding it for a whole litany of platforms detailed in the
changes to the docker files here. The existing `cross` image is also trimmed
down quite a bit to avoid duplicate work, and we'll eventually provision it for
far more cross compilation as well.
Note that the gcc toolchains installed to compile most of these compilers are
inappropriate for actualy distribution. The glibc they pull in is much newer
than we'd like, so before we turn nightlies off we'll need to tweak these docker
files to custom build toolchains like the current `linux-cross` docker image
does.
This commit adds six new travis matrix entires for doing cross-compiled
distribution builds of the compiler. The support added in #38731 allows us to
quickly compile a complete suite of distribution artifacts for cross-compiled
platforms, and currently each matrix entry (when fully cached) clocks in around
an hour to finish. Note that a full test run typically takes about two hours
right now.
With further optimizations coming down the pike in #39026 this commit also
starts doubling up cross-compiled distribution builders on each matrix entry. We
initially planned to do one build per entry, but it's looking like we may be
able to get by with more than one in each entry. Depending on how long these
builds take we may even be able to up it to three, but we'll start with two
first.
This commit then completes the suite of cross-compiled compilers that we're
going to compile, adding it for a whole litany of platforms detailed in the
changes to the docker files here. The existing `cross` image is also trimmed
down quite a bit to avoid duplicate work, and we'll eventually provision it for
far more cross compilation as well.
Note that the gcc toolchains installed to compile most of these compilers are
inappropriate for actualy distribution. The glibc they pull in is much newer
than we'd like, so before we turn nightlies off we'll need to tweak these docker
files to custom build toolchains like the current `linux-cross` docker image
does.
This commit configures local sccache directory to get used if you're running
builds locally to enjoy the benefits of caching when running inside the
containers.
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.
The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
This commit switches the default build system for Rust from the makefiles to
rustbuild. The rustbuild build system has been in development for almost a year
now and has become quite mature over time. This commit is an implementation of
the proposal on [internals] which slates deletion of the makefiles on
2016-01-02.
[internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368
This commit also updates various documentation in `README.md`,
`CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of
rustbuild itself.
Closes#37858
This commit configures our `.travis.yml` to test the full suite of tests we have
on Buildbot right now. A whole mess of docker images are added to the `src/ci`
directory which represent all the build environments for each configuration.
Each of these environments is then configured in `.travis.yml` to run on the
auto branch.
Note that the full matrix of tests aren't intended to be run on all PRs.
Instead, we continue to run only one entry in the matrix, forcing all others to
finish quickly. Only the `auto` branch should run the full matrix of builds.
Also note that the infrastructure hasn't quite been allocated yet to the
rust-lang/rust repository, so everything is disabled for now except for the one
build that happens on PRs. Once that infrastructure is allocated though we can
enable this and let it fly!
Notable modifications from the current test suite today:
* Android tests are run in rustbuild instead of the makefiles, for whatever
reason I couldn't get the makefiles to work on Travis.
* A debuginfo test was updated to work with the current version of the Android
NDK.
* Some dependencies in `mk/tests.mk` were fixed to allow running tests in
parallel.
