There's a trailing whitespace problem in one of the tests. @nrc said
he'll go fix it quickly, but until then I'll like to land this PR.
I suspect this happened because one of the dependencies in the
Cargo.lock I updated had changed the format of the XML they emit, but
that has to be investigated.
This commit alters how we compile LLVM by default enabling the WebAssembly
backend. This then also adds the wasm32-unknown-unknown target to get compiled
on the `cross` builder and distributed through rustup. Tests are not yet enabled
for this target but that should hopefully be coming soon!
This makes it more robust when assertions are disabled,
crashing instead of causing UB.
Also introduces a tidy check to enforce this rule,
which in turn necessitated making tidy run on src/rustllvm.
Fixes#44020
std: Add a new wasm32-unknown-unknown target
This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this target.
* Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new target.
This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready".
### Building yourself
First you'll need to configure the build of LLVM and enable this target
```
$ ./configure --target=wasm32-unknown-unknown --set llvm.experimental-targets=WebAssembly
```
Next you'll want to remove any previously compiled LLVM as it needs to be rebuilt with WebAssembly support. You can do that with:
```
$ rm -rf build
```
And then you're good to go! A `./x.py build` should give you a rustc with the appropriate libstd target.
### Test support
Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is [still getting LLVM bugs fixed](https://reviews.llvm.org/D39866) to get that working and will take some time. Relatively simple programs all seem to work though!
In general I've only tested this with a local fork that makes use of LLVM 5 rather than our current LLVM 4 on master. The LLVM 4 WebAssembly backend AFAIK isn't broken per se but is likely missing bug fixes available on LLVM 5. I'm hoping though that we can decouple the LLVM 5 upgrade and adding this wasm target!
### But the modules generated are huge!
It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
This commit adds a new target to the compiler: wasm32-unknown-unknown. This
target is a reimagining of what it looks like to generate WebAssembly code from
Rust. Instead of using Emscripten which can bring with it a weighty runtime this
instead is a target which uses only the LLVM backend for WebAssembly and a
"custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than
the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker
is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this
target.
* Most of the standard library is stubbed out to return an error, but anything
related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new
target.
This target is currently somewhat janky due to how linking works. The "linking"
is currently unconditional whole program LTO (aka LLVM is being used as a
linker). Naturally that means compiling programs is pretty slow! Eventually
though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can
act as a catalyst for further experimentation in Rust with WebAssembly. Breaking
changes are very likely to land to this target, so it's not recommended to rely
on it in any critical capacity yet. We'll let you know when it's "production
ready".
---
Currently testing-wise this target is looking pretty good but isn't complete.
I've got almost the entire `run-pass` test suite working with this target (lots
of tests ignored, but many passing as well). The `core` test suite is still
getting LLVM bugs fixed to get that working and will take some time. Relatively
simple programs all seem to work though!
---
It's worth nothing that you may not immediately see the "smallest possible wasm
module" for the input you feed to rustc. For various reasons it's very difficult
to get rid of the final "bloat" in vanilla rustc (again, a real linker should
fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various
integration points if you've got better ideas of how to approach them!
incr.comp.: Implement query result cache and use it to cache type checking tables.
This is a spike implementation of caching more than LLVM IR and object files when doing incremental compilation. At the moment, only the `typeck_tables_of` query is cached but MIR and borrow-check will follow shortly. The feature is activated by running with `-Zincremental-queries` in addition to `-Zincremental`, it is not yet active by default.
r? @nikomatsakis
This adds a "min-system-llvm-version" directive, so that a test can
indicate that it will either work with rust-llvm or with some minimal
system LLVM. This makes it simpler to write a test that requires an
LLVM patch that landed upstream and was then backported to rust-llvm.
compiletest: Fix a couple of test re-run issues
* Re-run rustdoc tests if rustdoc or htmldocck.py was updated.
* Put stamp files in the correct subdirectories to avoid clashes when
the file names match but the subdirectory doesn't.
Fix test case header parsing code in presence of multiple revisions.
The previous code would parse the TestProps, and then parse them again with a revision set, adding some elements (like aux_builds) a second time to the existing TestProps.
* Re-run rustdoc tests if rustdoc or htmldocck.py was updated.
* Put stamp files in the correct subdirectories to avoid clashes when
the file names match but the subdirectory doesn't.
The previous code would parse the TestProps, and then parse them
again with a revision set, adding some elements (like aux_builds)
a second time to the existing TestProps.
Miscellaneous changes for CI, Docker and compiletest.
This PR contains 7 independent commits that improves interaction with CI, Docker and compiletest.
1. a4e5c91cb8 — Forces a newline every 100 dots when testing in quiet mode. Prevents spurious timeouts when abusing the CI to test Android jobs.
2. 1b5aaf22e8 — Use vault.centos.org for dist-powerpc64le-linux, see #45744.
3. 33400fbbcd — Modify `src/ci/docker/run.sh` so that the docker images can be run from Docker Toolbox for Windows on Windows 7. I haven't checked the behavior of the newer Docker for Windows on Windows 10. Also, "can run" does not mean all the test can pass successfully (the UDP tests failed last time I checked)
4. d517668a08 — Don't emit a real warning the linker segfault, which affects UI tests like https://github.com/rust-lang/rust/pull/45489#issuecomment-340134944. Log it instead.
5. 51e2247948 — During run-pass, trim the output if stdout/stderr exceeds 416 KB (top 160 KB + bottom 256 KB). This is an attempt to avoid spurious failures like https://github.com/rust-lang/rust/pull/45384#issuecomment-341755788
6. 9cfdabaf3c — Force `gem update --system` before deploy. This is an attempt to prevent spurious error #44159.
7. eee10cc482 — Tries to print the crash log on macOS on failure. This is an attempt to debug #45230.
incr.comp.: Verify stability of incr. comp. hashes and clean up various other things.
The main contribution of this PR is that it adds the `-Z incremental-verify-ich` functionality. Normally, when the red-green tracking system determines that a certain query result has not changed, it does not re-compute the incr. comp. hash (ICH) for that query result because that hash is already known. `-Z incremental-verify-ich` tells the compiler to re-hash the query result and compare the new hash against the cached hash. This is a rather thorough way of
- testing hashing implementation stability,
- finding missing `[input]` annotations on `DepNodes`, and
- finding missing read-edges,
since both a missed read and a missing `[input]` annotation can lead to something being marked as green instead of red and thus will have a different hash than it should have.
Case in point, implementing this verification logic and activating it for all `src/test/incremental` tests has revealed several such oversights, all of which are fixed in this PR.
r? @nikomatsakis
tools: Fix rustfmt and the RLS
These tools have been corrected in their upstream repo's, and the submodules have been updated here to reflect that. I also had to update Cargo to match what the RLS is expecting.
The tool states for `rustfmt` and `rls` where both changed from "Broken" to "Testing" in this commit, thus enabling testing and distribution again.
These tools have been corrected in their upstream repo's, and the
submodules have been updated here to reflect that. I also had to update
Cargo to match what the RLS is expecting.
The tool states for `rustfmt` and `rls` where both changed from "Broken"
to "Testing" in this commit, thus enabling testing and distribution
again.
This is intended to prevent the spurious OOM error from
run-pass/rustc-rust-log.rs, by skipping the output in the middle when the
size is over 416 KB, so that the log output will not be overwhelmed.
Copy all `AsciiExt` methods to the primitive types directly in order to deprecate it later
**EDIT:** [this PR is ready now](https://github.com/rust-lang/rust/pull/44042#issuecomment-333883548). I edited this post to reflect the current status of discussion, which is (apart from code review) pretty much settled.
---
This is my current progress in order to prepare stabilization of #39658. As discussed there (and in #39659), the idea is to deprecated `AsciiExt` and copy all methods to the type directly. Apparently there isn't really a reason to have those methods in an extension trait¹.
~~This is **work in progress**: copy&pasting code while slightly modifying the documentation isn't the most exciting thing to do. Therefore I wanted to already open this WIP PR after doing basically 1/4 of the job (copying methods to `&[u8]`, `char` and `&str` is still missing) to get some feedback before I continue. Some questions possibly worth discussing:~~
1. ~~Does everyone agree that deprecating `AsciiExt` is a good idea? Does everyone agree with the goal of this PR?~~ => apparently yes
2. ~~Are my changes OK so far? Did I do something wrong?~~
3. ~~The issue of the unstable-attribute is currently set to 0. I would wait until you say "Ok" to the whole thing, then create a tracking issue and then insert the correct issue id. Is that ok?~~
4. ~~I tweaked `eq_ignore_ascii_case()`: it now takes the argument `other: u8` instead of `other: &u8`. The latter was enforced by the trait. Since we're not bound to a trait anymore, we can drop the reference, ok?~~ => I reverted this, because the interface has to match the `AsciiExt` interface exactly.
¹ ~~Could it be that we can't write `impl [u8] {}`? This might be the reason for `AsciiExt`. If that is the case: is there a good reason we can't write such an impl block? What can we do instead?~~ => we couldn't at the time this PR was opened, but Simon made it possible.
/cc @SimonSapin @zackw
Otherwise changes to the compiler are unable to introduce new
warnings: some crates tested by cargotest deny all warnings and
thus, the CI build fails.
Thanks SimonSapin for the patch!
Shorten paths to auxiliary files created by tests
I'm hitting issues with long file paths to object files created by the test suite, similar to https://github.com/rust-lang/rust/issues/45103#issuecomment-335622075.
If we look at the object file path in https://github.com/rust-lang/rust/issues/45103 we can see that the patch contains of few components:
```
specialization-cross-crate-defaults.stage2-x86_64-pc-windows-gnu.run-pass.libaux\specialization_cross_crate_defaults.specialization_cross_crate_defaults0.rust-cgu.o
```
=>
1. specialization-cross-crate-defaults // test name, required
2. stage2 // stage disambiguator, required
3. x86_64-pc-windows-gnu // target disambiguator, required
4. run-pass // mode disambiguator, rarely required
5. libaux // suffix, can be shortened
6. specialization_cross_crate_defaults // required, there may be several libraries in the directory
7. specialization_cross_crate_defaults0 // codegen unit name, can be shortened?
8. rust-cgu // suffix, can be shortened?
9. o // object file extension
This patch addresses items `4`, `5` and `8`.
`libaux` is shortened to `aux`, `rust-cgu` is shortened to `rcgu`, mode disambiguator is omitted unless it's necessary (for pretty-printing and debuginfo tests, see https://github.com/rust-lang/rust/pull/24537/commits/38d26d811a44ba93637c84ce77a58af88c47f0ac)
I haven't touched names of codegen units though (`specialization_cross_crate_defaults0`).
Is it useful for them to have descriptive names including the crate name, as opposed to just `0` or `cgu0` or something?
