This ensures that each build will support the testing design of "dry
running" builds. It's also checked that a dry run build is equivalent
step-wise to a "wet" run build; the graphs we generate when running are
directly compared node/node and edge/edge, both for order and contents.
This commit updates the `ToolBuild` step to stream Cargo's JSON messages, parse
them, and record all libraries built. If we build anything twice (aka Cargo)
it'll most likely happen due to dependencies being recompiled which is caught by
this check.
This commit disables building documentation on cross-compiled compilers, for
example ARM/MIPS/PowerPC/etc. Currently I believe we're not getting much use out
of these documentation artifacts and they often take 10-15 minutes total to
build as it requires building rustdoc/rustbook and then also generating all the
documentation, especially for the reference and the book itself.
In an effort to cut down on the amount of work that we're doing on dist CI
builders in light of recent timeouts this was some relatively low hanging fruit
to cut which in theory won't have much impact on the ecosystem in the hopes that
the documentation isn't used too heavily anyway.
While initial analysis in #48827 showed only shaving 5 minutes off local builds
the same 5 minute conclusion was drawn from #48826 which ended up having nearly
a half-hour impact on the bots. In that sense I'm hoping that we can land this
and test out what happens on CI to see how it affects timing.
Note that all tier 1 platforms, Windows, Mac, and Linux, will continue to
generate documentation.
This commit tweaks where timing and step information is printed out as part of
the build, ensuring that we do it as close to the location where work happens as
possible. In rustbuild various functions may perform long blocking work as
dependencies are assembled, so if we print out timing information early on we
may accidentally time more than just the step we were intending to time!
This reverts commit ab018c76e1.
This also adds the `ToolBuild::is_ext_tool` field to replace the previous
`ToolBuild::expectation` field, to indicate whether a build-failure of
certain tool is essential.
If build failed for these tools, they will be automatically skipped from
distribution, and will not fail the whole build.
Test failures are *not* ignored, nor build failure of other tools (e.g.
cargo). Therefore it should have no observable effect to the current CI
system.
This is step 1/8 of automatic management of broken tools #45861.
At stage 1 rustdoc is built at stage 0 so the error index generator should be as well.
This fixes `x.py --stage 1 doc` as rustdoc doesn't even build at stage 1.
We copy built tool binaries into a dedicated directory to avoid deleting
them, stageN-tools-bin. These aren't ever cleared out by code, since
there should be no reason to do so, and we'll simply overwrite them as
necessary.
When clearing out the stageN-{std,rustc,tools} directories, make sure to
delete both Cargo directories -- per-target and build scripts. This
ensures that changing libstd doesn't cause problems due to build scripts
not being rebuilt, even though they should be.
rustbuild: Fix test "test rustdoc" invocation
Previously it would use the librustc output directory which would cause rustdoc
to get entirely recompiled, whereas the intention is that it uses the
already-compiled artifacts from building rustdoc itself, using the tool output
directory
Previously it would use the librustc output directory which would cause rustdoc
to get entirely recompiled, whereas the intention is that it uses the
already-compiled artifacts from building rustdoc itself, using the tool output
directory
Right now we comiple rustdoc in stage 2 and the error index in stage 0, which
ends up compiling rustdoc twice! To avoid compiling rustdoc twice (which takes
awhile) let's just compile it once in stage 2.
The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.
This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.
Hopefully this can help speed up local builds by doing fewer copies all over the
place!
Update rls
And expose the `CFG_VERSION` env var to tools so they can determine the version of Rust.
This gets the RLS back on master and so completes the PR dance for the generators PR.
r? @alexcrichton
Add clippy as a submodule
~~This builds clippy as part of `./x.py build` (locally and in CI).~~
This allows building clippy with `./x.py build src/tools/clippy`
~~Needs https://github.com/nrc/dev-tools-team/issues/18#issuecomment-322456461 to be resolved before it can be merged.~~ Contributers can simply open a PR to clippy and point the submodule at the `pull/$pr_number/head` branch.
This does **not** build clippy or test the clippy test suite at all as per https://github.com/nrc/dev-tools-team/issues/18#issuecomment-321411418
r? @nrc
cc @Manishearth @llogiq @mcarton @alexcrichton
This fixes the bug we previously had where we'd build a libtest tool
after building a libstd tool and clear out the libstd tool. Since we
clear out all tools for a given stage on invocations of CleanTools after
lib{std, test, rustc} change, we need to make sure that all tools built
with that stage will be built after the clearing is done.
The fix contained here technically isn't perfect; there is still an edge
case of compiling a libstd tool, then compiling libtest, which will
clear out the libstd tool and it won't ever get rebuilt within that
session of rustbuild. This is where the caching system used today shows
it's problems -- in effect, all tools depend on a global counter of the
stage being cleared out. We can implement such a counter in a future
patch to ensure that tools are rebuilt as needed, but it is deemed
unlikely that it will be required in practice, since most if not all
tools are built after the relevant stage's std/test/rustc are built,
though this is only an opinion and hasn't been verified.
