travis: Gate on some minimal support for incremental compilation.
This commit adds a travis job that
1. builds a stage2 compiler in incremental mode (but with empty incremental compilation cache), and
2. builds and runs the run-pass test suite also in incremental mode.
Building incrementally with an empty cache makes sure that the compiler doesn't crash in dependency tracking during bootstrapping. Executing the incrementally built test suite gives some measure of confidence that we generate valid code.
Note, however, that the above does not give strong guarantees about the validity of incremental compilation, it just provides a basis for being able to rely on from-scratch incr. comp. builds as reference values in further tests (which then do actual incremental compilation).
r? @alexcrichton
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
rustbuild: Add manifest generation in-tree
This commit adds a new tool, `build-manifest`, which is used to generate a
distribution manifest of all produced artifacts. This tool is intended to
replace the `build-rust-manifest.py` script that's currently located on the
buildmaster. The intention is that we'll have a builder which periodically:
* Downloads all artifacts for a commit
* Runs `./x.py dist hash-and-sign`. This will generate `sha256` and `asc` files
as well as TOML manifests.
* Upload all generated hashes and manifests to the directory the artifacts came
from.
* Upload *all* artifacts (tarballs and hashes and manifests) to an archived
location.
* If necessary, upload all artifacts to the main location.
This script is intended to just be the second step here where orchestrating
uploads and such will all happen externally from the build system itself.
cc #38531
This commit adds a new tool, `build-manifest`, which is used to generate a
distribution manifest of all produced artifacts. This tool is intended to
replace the `build-rust-manifest.py` script that's currently located on the
buildmaster. The intention is that we'll have a builder which periodically:
* Downloads all artifacts for a commit
* Runs `./x.py dist hash-and-sign`. This will generate `sha256` and `asc` files
as well as TOML manifests.
* Upload all generated hashes and manifests to the directory the artifacts came
from.
* Upload *all* artifacts (tarballs and hashes and manifests) to an archived
location.
* If necessary, upload all artifacts to the main location.
This script is intended to just be the second step here where orchestrating
uploads and such will all happen externally from the build system itself.
PR #38842 has exposed that we were missing the src/test/compile-fail-fulldeps
directory in the search for feature gate tests. Because the detection didn't
work despite the effort to name the test appropriately and add a correct
"// gate-test-proc_macro" comment, proc_macro was added to the whitelist.
We fix this little weakness in the feature gate tidy check and add
the src/test/compile-fail-fulldeps directory to the checked directories.
linkchecker: Fix checking links which are just fragments
Also fix a typo which linkchecker should have caught.
It was broken by 31a8638e5e.
r? @alexcrichton
travis: Add i586 linux and i686 musl
This commit expands the existing x86_64-musl entry in the Travis matrix to also
build/test i586-unknown-linux-gnu and i686-unknown-linux-musl.
cc #38531Closes#35599Closes#39053
Implement `#[proc_macro_attribute]`
This implements `#[proc_macro_attribute]` as described in https://github.com/rust-lang/rfcs/pull/1566
The following major (hopefully non-breaking) changes are included:
* Refactor `proc_macro::TokenStream` to use `syntax::tokenstream::TokenStream`.
* `proc_macro::tokenstream::TokenStream` no longer emits newlines between items, this can be trivially restored if desired
* `proc_macro::TokenStream::from_str` does not try to parse an item anymore, moved to `impl MultiItemModifier for CustomDerive` with more informative error message
* Implement `#[proc_macro_attribute]`, which expects functions of the kind `fn(TokenStream, TokenStream) -> TokenStream`
* Reactivated `#![feature(proc_macro)]` and gated `#[proc_macro_attribute]` under it
* `#![feature(proc_macro)]` and `#![feature(custom_attribute)]` are mutually exclusive
* adding `#![feature(proc_macro)]` makes the expansion pass assume that any attributes that are not built-in, or introduced by existing syntax extensions, are proc-macro attributes
* Fix `feature_gate::find_lang_feature_issue()` to not use `unwrap()`
* This change wasn't necessary for this PR, but it helped debugging a problem where I was using the wrong feature string.
* Move "completed feature gate checking" pass to after "name resolution" pass
* This was necessary for proper feature-gating of `#[proc_macro_attribute]` invocations when the `proc_macro` feature flag isn't set.
Prototype/Litmus Test: [Implementation](https://github.com/abonander/anterofit/blob/proc_macro/service-attr/src/lib.rs#L13) -- [Usage](https://github.com/abonander/anterofit/blob/proc_macro/service-attr/examples/post_service.rs#L35)
* Add support for `#[proc_macro]`
* Reactivate `proc_macro` feature and gate `#[proc_macro_attribute]` under it
* Have `#![feature(proc_macro)]` imply `#![feature(use_extern_macros)]`,
error on legacy import of proc macros via `#[macro_use]`
This commit expands the existing x86_64-musl entry in the Travis matrix to also
build/test i586-unknown-linux-gnu and i686-unknown-linux-musl.
cc #38531Closes#39053
This marks the pushpop_unsafe feature as removed inside the feature_gate.
It was added in commit 1829fa5199 and then
removed again in commit d399098fd8 .
Seems that the second commit forgot to mark it as removed in feature_gate.rs.
This enables us to remove another element from the whitelist of non gate
tested unstable lang features (issue #39059).
This removes the safe_suggestion feature from feature_gate.rs.
It was added in commit 164f0105bb
and then removed again in commit c11fe553df .
As the removal was in the same PR #38099 as the addition, we don't move it to
the "removed" section.
Removes an element from the whitelist of non gate tested unstable lang features (issue #39059).
Its non trivial to test lang feature gates, and people
forget to add such tests. So we extend the features lint
of the tidy tool to ensure that all new lang features
contain a new compile-fail test.
Of course, one could drop this requirement and just
grep all tests in run-pass for #![feature(abc)] and
then run this test again, removing the mention,
requiring that it fails.
But this only tests for the existence of a compilation
failure. Manual tests ensure that also the correct lines
spawn the error, and also test the actual error message.
For library features, it makes no sense to require such
a test, as here code is used that is generic for all
library features.
Local testing showed that I was able to reproduce an error where debuginfo tests
on Android would fail with "connection reset by peer". Further investigation
turned out that the gdb tests are android with bit of process management:
* First an `adb forward` command is run to ensure that the host's port 5039 is
the same as the emulator's.
* Next an `adb shell` command is run to execute the `gdbserver` executable
inside the emulator. This gdb server will attach to port 5039 and listen for
remote gdb debugging sessions.
* Finally, we run `gdb` on the host (not in the emulator) and then connect to
this gdb server to send it commands.
The problem was happening when the host's gdb was failing to connect to the
remote gdbserver running inside the emulator. The previous test for this was
that after `adb shell` executed we'd sleep for a second and then attempt to make
a TCP connection to port 5039. If successful we'd run gdb and on failure we'd
sleep again.
It turns out, however, that as soon as we've executed `adb forward` all TCP
connections to 5039 will succeed. This means that we would only ever sleep for
at most one second, and if this wasn't enough time we'd just fail later because
we would assume that gdbserver had started but it may not have done so yet.
This commit fixes these issues by removing the TCP connection to test if
gdbserver is ready to go. Instead we read the stdout of the process and wait for
it to print that it's listening at which point we start running gdb. I've found
that locally at least I was unable to reproduce the failure after these changes.
Closes#38710
i128 and u128 support
Brings i128 and u128 support to nightly rust, behind a feature flag. The goal of this PR is to do the bulk of the work for 128 bit integer support. Smaller but just as tricky features needed for stabilisation like 128 bit enum discriminants are left for future PRs.
Rebased version of #37900, which in turn was a rebase + improvement of #35954 . Sadly I couldn't reopen#37900 due to github. There goes my premium position in the homu queue...
[plugin-breaking-change]
cc #35118 (tracking issue)
cargotest: Add xsv to tested crates
This was intended to land in #37149 but I ended up backing it out to land the
rollup (#38697) last night as I was itching to do so. This morning though xsv
has been fixed now (BurntSushi/xsv#53) so we should be able to add it!
This was intended to land in #37149 but I ended up backing it out to land the
rollup (#38697) last night as I was itching to do so. This morning though xsv
has been fixed now (BurntSushi/xsv#53) so we should be able to add it!
This commit changes all tools and such to get compiled in stage0, not in
later stages. The purpose of this commit is to cut down dependencies on later
stages for future modifications to the build system. Notably we're going to be
adding builders that produce a full suite of cross-compiled artifacts for a
particular host, and that shouldn't compile the `x86_64-unknown-linux-gnu`
compiler more than once. Currently dependencies on, for example, the error index
end up compiling the `x86_64-unknown-linux-gnu` compiler more than necessary.
As a result here we move many dependencies on these tools to being produced by a
stage0 compiler, not a stage1+ compiler. None of these tools actually need to be
staged at all, so they'll exhibit consistent behavior across the stages.
The check inside compiler-rt file int_types.h to #define CRT_HAS_128BIT
looks like:
#if (defined(__LP64__) || defined(__wasm__)) && \
!(defined(__mips__) && defined(__clang__))
#define CRT_HAS_128BIT
#endif
Windows uses LLP64 instead of LP64, so it doesn't ship with the C based
intrinsics.
Also, add libcompiler_builtins to the list of crates that may have platform
specific checks (like the ones we just added).
appveyor: Attempt to debug flaky test runs
This commit is an attempt to debug #38620 since we're unable to reproduce it
locally. It follows the [advice] of those with AppVeyor to use the `handle.exe`
tool to try to debug what processes have a handle to the file open.
This won't be guaranteed to actually help us, but hopefully it'll diagnose
something at some point?
[advice]: http://help.appveyor.com/discussions/questions/2898
compiletest: Don't limit all suites on Android
On Android we only have one test thread for supposed problems with concurrency
and the remote debugger. Not all of our suites require one concurrency, however,
and suites like compile-fail or pretty can be much faster if they're
parallelized on Travis.
This commit only sets the test threads to one on Android for suites which
actually run code, and other suites aren't tampered with.
Add some more repos to cargotest
From suggestions at https://users.rust-lang.org/t/what-stable-rust-applications-do-you-use-frequently/7618
This adds some applications which use stable Rust and come with their own lockfiles in their respective trees.
ripgrep, xsv, and bins have 33 unique dependencies between them.
I alphabetized the list by project name because that seems tidier.
r? @brson
This commit is an attempt to debug #38620 since we're unable to reproduce it
locally. It follows the [advice] of those with AppVeyor to use the `handle.exe`
tool to try to debug what processes have a handle to the file open.
This won't be guaranteed to actually help us, but hopefully it'll diagnose
something at some point?
[advice]: http://help.appveyor.com/discussions/questions/2898
* Update to ripgrep HEAD because the previous rev would still change the lock
file when `cargo build` was issued.
* Remove `bins` as it depends on OpenSSL on Windows, which won't work on our
bots
* Update rev of tokei to get a rev that doesn't change the lockfile
On Android we only have one test thread for supposed problems with concurrency
and the remote debugger. Not all of our suites require one concurrency, however,
and suites like compile-fail or pretty can be much faster if they're
parallelized on Travis.
This commit only sets the test threads to one on Android for suites which
actually run code, and other suites aren't tampered with.
A tweak was made to dependencies in #38451 but the makefiles weren't updated to
accompany this. Instead of trying to integerate the `build_helper` crate into
the makefiles (which currently isn't present) this commit takes the approach of
just duplicating the required logic, which should be small enough for now.
adaptation to rustbuild for openbsd
Since the switch to rustbuild, the build for openbsd is broken:
- [X] `ar` inference based on compiler name is wrong (OpenBSD usually use `egcc`, but `ear` doesn't exist)
- [X] `make` isn't GNU-make under OpenBSD (and others BSD platforms)
- [x] `stdc++` isn't the right stdc++ library to link with (it should be `estdc++`)
- [x] corrects tests that don't pass anymore (problems related to rustbuild)
r? @alexcrichton
A more verbose matching failure for mir tests
This makes it easier to work with mir test failures during development.
- Show which expected line was not found
- Show full expected output
- Show full actual output
