bootstrap: Fix all the pep-8 issues reported by flake8
This commit also adds a few missing docstrings.
Today, after reading this [article](https://blog.rust-lang.org/2017/06/27/Increasing-Rusts-Reach.html), I downloaded this project and started building from source. In the meantime, I began to read the `bootstrap.py`, to know more about the building process, and I made a few changes, this is my first contribution to the project, hope you like it.
BTW, I have a few doubts about the `bootstrap.py`, any guidance is more than welcome:
* Where can I find the unit tests for this script? In case it doesn't exist yet, do you like to include some unit tests with pytest?
* Some methods like `fix_executable`, `get_string`, and `exe_suffix` in the `RustBuild` class should be converted to a function because it doesn't use `self` anywhere. What do you think?
Make wasm32 buildbot test LLVM backend
This adds the experimental targets option to configure so it can be used
by the builders and changes the wasm32 Dockerfile accordingly. Instead
of using LLVM from the emsdk, the builder's emscripten tools now uses
the Rust in-tree LLVM, since this is the one built with wasm support.
This adds the experimental targets option to configure so it can be used
by the builders and changes the wasm32 Dockerfile accordingly. Instead
of using LLVM from the emsdk, the builder's emscripten tools now uses
the Rust in-tree LLVM, since this is the one built with wasm support.
Pass path to python from bootstrap.py to bootstrap.rs
When bootstrap is executed with python not in `$PATH`, (e. g.
`c:\Python27\python.exe x.py test`) bootstrap cannot find python
and crashes.
This commit passes path to python in `BOOTSTRAP_PYTHON` env var.
Make rustc errors colorful.
Rustbuild passes --message-format=json to Cargo to learn about the
dependencies for a given build, which then makes Cargo steal the
stderr/stdout for the compiler process, leading to non colorful output.
To avoid this, detection of stderr being a tty is added to rustbuild,
and an environment variable is used to communicate with the rustc shim.
Fixes https://github.com/rust-lang/rust/issues/42801.
r? @alexcrichton
Fixes bootstrapping with custom cargo/rustc.
config.mk is now always read when parsing the configuration to prevent
this from reoccurring in the future, hopefully.
Fixes https://github.com/rust-lang/rust/issues/42543.
r? @alexcrichton
cc @infinity0 @kyrias
Rustbuild passes --message-format=json to Cargo to learn about the
dependencies for a given build, which then makes Cargo steal the
stderr/stdout for the compiler process, leading to non colorful output.
To avoid this, detection of stderr being a tty is added to rustbuild,
and an environment variable is used to communicate with the rustc shim.
When bootstrap is executed with python not in `$PATH`, (e. g.
`c:\Python27\python.exe x.py test`) bootstrap cannot find python
and crashes.
This commit passes path to python in `BOOTSTRAP_PYTHON` env var.
This commit deletes the in-tree `getopts` crate in favor of the crates.io-based
`getopts` crate. The main difference here is with a new builder-style API, but
otherwise everything else remains relatively standard.
Enable wasm LLVM backend
Enables compilation to WebAssembly with the LLVM backend using the target triple "wasm32-unknown-unknown". This is the beginning of my work on #38804.
**edit:** The new new target is now wasm32-experimental-emscripten instead of wasm32-unknown-unknown.
Use custom cargo/rustc paths when parsing flags.
Fixes https://github.com/rust-lang/rust/issues/41779, probably also https://github.com/rust-lang/rust/issues/42543 (I think they're duplicates).
I'm not entirely happy with the implementation, since it means we parse the configuration twice, but it's the minimal solution. I think the other choice is to move both calls to Config::parse inside Flags::parse and merge them, but I don't know if that's a good idea.
r? @alexcrichton
The new target is wasm32-experimental-emscripten. Adds a new
configuration option to opt in to building experimental LLVM backends
such as the WebAssembly backend. The target name was chosen to be
similar to the existing wasm32-unknown-emscripten target so that the
build and tests would work with minimal other code changes. When/if the
new target replaces the old target, simply renaming it should just work.
Autogenerate stubs and SUMMARY.md in the unstable book
Removes a speed bump in compiler development by autogenerating stubs for features in the unstable book. See #42454 for discussion.
The PR contains three commits, separated in order to make review easy:
* The first commit converts the tidy tool from a binary crate to a crate that contains both a library and a binary. In the second commit, we'll use the tidy library
* The second and main commit introduces autogeneration of SUMMARY.md and feature stub files
* The third commit turns off the tidy lint that checks for features without a stub, and removes the stub files. A separate commit due to the large number of files touched
Members of the doc team who wish to document some features can either do this (where `$rustsrc` is the root of the rust repo git checkout):
1. cd to `$rustsrc/src/tools/unstable-book-gen` and then do `cargo run $rustsrc/src $rustsrc/src/doc/unstable-book` to put the stubs into the unstable book
2. cd to `$rustsrc` and run `git ls-files --others --exclude-standard` to list the newly added stubs
3. choose a file to edit, then `git add` it and `git commit`
4. afterwards, remove all changes by the tool by doing `git --reset hard` and `git clean -f`
Or they can do this:
1. remove the comment marker in `src/tools/tidy/src/unstable_book.rs` line 122
2. run `./x.py test src/tools/tidy` to list the unstable features which only have stubs
3. revert the change in 1
3. document one of the chosen unstable features
The changes done by this PR also allow for further development:
* tidy obtains information about tracking issues. We can now forbid differing tracking issues between differing `#![unstable]` annotations. I haven't done this but plan to in a future PR
* we now have a general framework for generating stuff for the unstable book at build time. Further changes can autogenerate a list of the API a given library feature exposes.
The old way to simply click through the documentation after it has been uploaded to rust-lang.org works as well.
r? @nagisa
Fixes#42454
Add a travis builder for wasm32-unknown-emscripten
This commits add an entry to travis matrix that will execute wasm32-unknown-emscripten tests suites.
- Emscripten for asmjs was updated to sdk-1.37.13-64bit
- The tests are run with node 8.0.0 (it can execute wasm)
- A wrapper script is used to run each test from the directory where it is (workaround for https://github.com/kripken/emscripten/issues/4542)
- Some tests are ignore, see #42629 and #42630
Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
bootstrap: improve 'build --help' by explaining what exactly the last example does
I recently found myself confused about what exactly gets built how often when I run which command; I think this would have helped me.
One thing I did not touch, but I feel could also be improved, is the wording just above: "For a quick build with a usable compile, you can pass". I am not a native speaker, but this sounds odd to me. Do you mean "For a quick build of a usable compiler" (but then it should say where the usable compiler is produced)? Or do you mean "For a quick build testing if the compiler is usable"? I can reword this, but I'd like to make sure I understand the intent of the message.
What about
```
For a quick build of a usable compiler, you can pass:
./x.py build --stage 1 src/libtest
This will first build everything once (like --stage 0 without further
arguments would), and then use the compiler built in stage 0 to build
src/libtest and its dependencies.
Once this is done, build/$ARCH/stage1 contains a usable compiler.
```
However, I am not sure this is actually true. In particular, why even bother building the libstd in stage 1? AFAIK that ends up in `build/*/stage1-std`, not in `build/*/stage1` (which is filled from `build/*/stage0-*`).
