It now says '#[feature] may not be used on the stable release channel'.
I had to convert this error from a lint to a normal compiler error.
I left the lint previously-used for this in place since removing it is
a breaking change. It will just go unused until the end of time.
Fixes#24125
Environment variables are global state so this can lead to surprising results if
the driver is called in a multithreaded environment (e.g. doctests). There
shouldn't be any memory corruption that's possible, but a lot of the bots have
been failing because they can't find `cc` or `gcc` in the path during doctests,
and I highly suspect that it is due to the compiler modifying `PATH` in a
multithreaded fashion.
This commit moves the logic for appending to `PATH` to only affect the child
process instead of also affecting the parent, at least for the linking stage.
When loading dynamic libraries the compiler still modifies `PATH` on Windows,
but this may be more difficult to fix than spawning off a new process.
It now says '#[feature] may not be used on the stable release channel'.
I had to convert this error from a lint to a normal compiler error.
I left the lint previously-used for this in place since removing it is
a breaking change. It will just go unused until the end of time.
Fixes#24125
Environment variables are global state so this can lead to surprising results if
the driver is called in a multithreaded environment (e.g. doctests). There
shouldn't be any memory corruption that's possible, but a lot of the bots have
been failing because they can't find `cc` or `gcc` in the path during doctests,
and I highly suspect that it is due to the compiler modifying `PATH` in a
multithreaded fashion.
This commit moves the logic for appending to `PATH` to only affect the child
process instead of also affecting the parent, at least for the linking stage.
When loading dynamic libraries the compiler still modifies `PATH` on Windows,
but this may be more difficult to fix than spawning off a new process.
`driver::build_output_filenames` calls `file_stem` on a PathBuf obtained from the output file compiler flag. It's possible to pass the empty string to this compiler flag. When file_stem is called on an empty Path, it returns None, which is unwrapped and the compiler panics.
This change modifies the `unwrap` to an `unwrap_or` so that the empty string is passed through the compilation pipeline until it reaches `trans:🔙:write_output_file`, which will emit an appropriate error.
Instead of panicking, the error that is emitted now is:
```
$ rustc -o "" thing.rs
error: could not write output to : No such file or directory
```
The `:` is a little strange, but it /is/ reporting the filename (the empty string) correctly, I suppose. Both gcc and clang hand the output file to ld, which emits a similar error message when faced with the empty string as an output file:
```
$ clang -o "" thing.c
ld: can't open output file for writing: , errno=2 for architecture x86_64
```
This PR also adds a test for this, in `run-make`. This fixes issue #26092.
Special thanks to @retep998 for the [excellent writeup](https://github.com/rust-lang/rfcs/issues/1061) of tasks to be done and @ricky26 for initially blazing the trail here!
# MSVC Support
This goal of this series of commits is to add MSVC support to the Rust compiler
and build system, allowing it more easily interoperate with Visual Studio
installations and native libraries compiled outside of MinGW.
The tl;dr; of this change is that there is a new target of the compiler,
`x86_64-pc-windows-msvc`, which will not interact with the MinGW toolchain at
all and will instead use `link.exe` to assemble output artifacts.
## Why try to use MSVC?
With today's Rust distribution, when you install a compiler on Windows you also
install `gcc.exe` and a number of supporting libraries by default (this can be
opted out of). This allows installations to remain independent of MinGW
installations, but it still generally requires native code to be linked with
MinGW instead of MSVC. Some more background can also be found in #1768 about the
incompatibilities between MinGW and MSVC.
Overall the current installation strategy is quite nice so long as you don't
interact with native code, but once you do the usage of a MinGW-based `gcc.exe`
starts to get quite painful.
Relying on a nonstandard Windows toolchain has also been a long-standing "code
smell" of Rust and has been slated for remedy for quite some time now. Using a
standard toolchain is a great motivational factor for improving the
interoperability of Rust code with the native system.
## What does it mean to use MSVC?
"Using MSVC" can be a bit of a nebulous concept, but this PR defines it as:
* The build system for Rust will build as much code as possible with the MSVC
compiler, `cl.exe`.
* The build system will use native MSVC tools for managing archives.
* The compiler will link all output with `link.exe` instead of `gcc.exe`.
None of these are currently implemented today, but all are required for the
compiler to fluently interoperate with MSVC.
## How does this all work?
At the highest level, this PR adds a new target triple to the Rust compiler:
x86_64-pc-windows-msvc
All logic for using MSVC or not is scoped within this triple and code can
conditionally build for MSVC or MinGW via:
#[cfg(target_env = "msvc")]
It is expected that auto builders will be set up for MSVC-based compiles in
addition to the existing MinGW-based compiles, and we will likely soon start
shipping MSVC nightlies where `x86_64-pc-windows-msvc` is the host target triple
of the compiler.
# Summary of changes
Here I'll explain at a high level what many of the changes made were targeted
at, but many more details can be found in the commits themselves. Many thanks to
@retep998 for the excellent writeup in rust-lang/rfcs#1061 and @rick26 for a lot
of the initial proof-of-concept work!
## Build system changes
As is probably expected, a large chunk of this PR is changes to Rust's build
system to build with MSVC. At a high level **it is an explicit non goal** to
enable building outside of a MinGW shell, instead all Makefile infrastructure we
have today is retrofitted with support to use MSVC instead of the standard MSVC
toolchain. Some of the high-level changes are:
* The configure script now detects when MSVC is being targeted and adds a number
of additional requirements about the build environment:
* The `--msvc-root` option must be specified or `cl.exe` must be in PATH to
discover where MSVC is installed. The compiler in use is also required to
target x86_64.
* Once the MSVC root is known, the INCLUDE/LIB environment variables are
scraped so they can be reexported by the build system.
* CMake is required to build LLVM with MSVC (and LLVM is also configured with
CMake instead of the normal configure script).
* jemalloc is currently unconditionally disabled for MSVC targets as jemalloc
isn't a hard requirement and I don't know how to build it with MSVC.
* Invocations of a C and/or C++ compiler are now abstracted behind macros to
appropriately call the underlying compiler with the correct format of
arguments, for example there is now a macro for "assemble an archive from
objects" instead of hard-coded invocations of `$(AR) crus liboutput.a ...`
* The output filenames for standard libraries such as morestack/compiler-rt are
now "more correct" on windows as they are shipped as `foo.lib` instead of
`libfoo.a`.
* Rust targets can now depend on native tools provided by LLVM, and as you'll
see in the commits the entire MSVC target depends on `llvm-ar.exe`.
* Support for custom arbitrary makefile dependencies of Rust targets has been
added. The MSVC target for `rustc_llvm` currently requires a custom `.DEF`
file to be passed to the linker to get further linkages to complete.
## Compiler changes
The modifications made to the compiler have so far largely been minor tweaks
here and there, mostly just adding a layer of abstraction over whether MSVC or a
GNU-like linker is being used. At a high-level these changes are:
* The section name for metadata storage in dynamic libraries is called `.rustc`
for MSVC-based platorms as section names cannot contain more than 8
characters.
* The implementation of `rustc_back::Archive` was refactored, but the
functionality has remained the same.
* Targets can now specify the default `ar` utility to use, and for MSVC this
defaults to `llvm-ar.exe`
* The building of the linker command in `rustc_trans:🔙:link` has been
abstracted behind a trait for the same code path to be used between GNU and
MSVC linkers.
## Standard library changes
Only a few small changes were required to the stadnard library itself, and only
for minor differences between the C runtime of msvcrt.dll and MinGW's libc.a
* Some function names for floating point functions have leading underscores, and
some are not present at all.
* Linkage to the `advapi32` library for crypto-related functions is now
explicit.
* Some small bits of C code here and there were fixed for compatibility with
MSVC's cl.exe compiler.
# Future Work
This commit is not yet a 100% complete port to using MSVC as there are still
some key components missing as well as some unimplemented optimizations. This PR
is already getting large enough that I wanted to draw the line here, but here's
a list of what is not implemented in this PR, on purpose:
## Unwinding
The revision of our LLVM submodule [does not seem to implement][llvm] does not
support lowering SEH exception handling on the Windows MSVC targets, so
unwinding support is not currently implemented for the standard library (it's
lowered to an abort).
[llvm]: https://github.com/rust-lang/llvm/blob/rust-llvm-2015-02-19/lib/CodeGen/Passes.cpp#L454-L461
It looks like, however, that upstream LLVM has quite a bit more support for SEH
unwinding and landing pads than the current revision we have, so adding support
will likely just involve updating LLVM and then adding some shims of our own
here and there.
## dllimport and dllexport
An interesting part of Windows which MSVC forces our hand on (and apparently
MinGW didn't) is the usage of `dllimport` and `dllexport` attributes in LLVM IR
as well as native dependencies (in C these correspond to
`__declspec(dllimport)`).
Whenever a dynamic library is built by MSVC it must have its public interface
specified by functions tagged with `dllexport` or otherwise they're not
available to be linked against. This poses a few problems for the compiler, some
of which are somewhat fundamental, but this commit alters the compiler to attach
the `dllexport` attribute to all LLVM functions that are reachable (e.g. they're
already tagged with external linkage). This is suboptimal for a few reasons:
* If an object file will never be included in a dynamic library, there's no need
to attach the dllexport attribute. Most object files in Rust are not destined
to become part of a dll as binaries are statically linked by default.
* If the compiler is emitting both an rlib and a dylib, the same source object
file is currently used but with MSVC this may be less feasible. The compiler
may be able to get around this, but it may involve some invasive changes to
deal with this.
The flipside of this situation is that whenever you link to a dll and you import
a function from it, the import should be tagged with `dllimport`. At this time,
however, the compiler does not emit `dllimport` for any declarations other than
constants (where it is required), which is again suboptimal for even more
reasons!
* Calling a function imported from another dll without using `dllimport` causes
the linker/compiler to have extra overhead (one `jmp` instruction on x86) when
calling the function.
* The same object file may be used in different circumstances, so a function may
be imported from a dll if the object is linked into a dll, but it may be
just linked against if linked into an rlib.
* The compiler has no knowledge about whether native functions should be tagged
dllimport or not.
For now the compiler takes the perf hit (I do not have any numbers to this
effect) by marking very little as `dllimport` and praying the linker will take
care of everything. Fixing this problem will likely require adding a few
attributes to Rust itself (feature gated at the start) and then strongly
recommending static linkage on Windows! This may also involve shipping a
statically linked compiler on Windows instead of a dynamically linked compiler,
but these sorts of changes are pretty invasive and aren't part of this PR.
## CI integration
Thankfully we don't need to set up a new snapshot bot for the changes made here as our snapshots are freestanding already, we should be able to use the same snapshot to bootstrap both MinGW and MSVC compilers (once a new snapshot is made from these changes).
I plan on setting up a new suite of auto bots which are testing MSVC configurations for now as well, for now they'll just be bootstrapping and not running tests, but once unwinding is implemented they'll start running all tests as well and we'll eventually start gating on them as well.
---
I'd love as many eyes on this as we've got as this was one of my first interactions with MSVC and Visual Studio, so there may be glaring holes that I'm missing here and there!
cc @retep998, @ricky26, @vadimcn, @klutzy
r? @brson
It looks like section names in objects generated by `link.exe` are limited to at
most 8 characters in length, so shorten `.note.rustc` to just `.rustc`
This allows compiling entire crates from memory or preprocessing source files before they are tokenized.
Minor API refactoring included, which is a [breaking-change] for libsyntax users:
* `ParseSess::{next_node_id, reserve_node_ids}` moved to rustc's `Session`
* `new_parse_sess` -> `ParseSess::new`
* `new_parse_sess_special_handler` -> `ParseSess::with_span_handler`
* `mk_span_handler` -> `SpanHandler::new`
* `default_handler` -> `Handler::new`
* `mk_handler` -> `Handler::with_emitter`
* `string_to_filemap(sess source, path)` -> `sess.codemap().new_filemap(path, source)`
Fixes the problem in #16974 with unhelpful error messages when accidentally using the wrong syntax for the `crate_type="lib"` attribute. The attribute syntax error now shows up instead of "main function not found".
If the user intended to set the crate_type to "lib" but accidentally used
incorrect syntax such as `#![crate_type(lib)]`, the compilation would fail with
"main function not found". This made it hard to locate the source of the
problem, since the failure would cause the warning about the incorrect
attribute not to be shown.
Turns out that a verbatim path was leaking through to gcc via the PATH
environment variable (pointing to the bundled gcc provided by the main
distribution) which was wreaking havoc when gcc itself was run. The fix here is
to just stop passing verbatim paths down by adding more liberal uses of
`fix_windows_verbatim_for_gcc`.
Closes#25072
These implementations were intended to be unstable, but currently the stability
attributes cannot handle a stable trait with an unstable `impl` block. This
commit also audits the rest of the standard library for explicitly-`#[unstable]`
impl blocks. No others were removed but some annotations were changed to
`#[stable]` as they're defacto stable anyway.
One particularly interesting `impl` marked `#[stable]` as part of this commit
is the `Add<&[T]>` impl for `Vec<T>`, which uses `push_all` and implicitly
clones all elements of the vector provided.
Closes#24791
I've been working on improving the diagnostic registration system so that it can:
* Check uniqueness of error codes *across the whole compiler*. The current method using `errorck.py` is prone to failure as it relies on simple text search - I found that it breaks when referencing an error's ident within a string (e.g. `"See also E0303"`).
* Provide JSON output of error metadata, to eventually facilitate HTML output, as well as tracking of which errors need descriptions. The current schema is:
```
<error code>: {
"description": <long description>,
"use_site": {
"filename": <filename where error is used>,
"line": <line in file where error is used>
}
}
```
[Here's][metadata-dump] a pretty-printed sample dump for `librustc`.
One thing to note is that I had to move the diagnostics arrays out of the diagnostics modules. I really wanted to be able to capture error usage information, which only becomes available as a crate is compiled. Hence all invocations of `__build_diagnostics_array!` have been moved to the ends of their respective `lib.rs` files. I tried to avoid moving the array by making a plugin that expands to nothing but couldn't invoke it in item position and gave up on hackily generating a fake item. I also briefly considered using a lint, but it seemed like it would impossible to get access to the data stored in the thread-local storage.
The next step will be to generate a web page that lists each error with its rendered description and use site. Simple mapping and filtering of the metadata files also allows us to work out which error numbers are absent, which errors are unused and which need descriptions.
[metadata-dump]: https://gist.github.com/michaelsproul/3246846ff1bea71bd049
Closes#17841.
The majority of the work should be done, e.g. trait and inherent impls, different forms of UFCS syntax, defaults, and cross-crate usage. It's probably enough to replace the constants in `f32`, `i8`, and so on, or close to good enough.
There is still some significant functionality missing from this commit:
- ~~Associated consts can't be used in match patterns at all. This is simply because I haven't updated the relevant bits in the parser or `resolve`, but it's *probably* not hard to get working.~~
- Since you can't select an impl for trait-associated consts until partway through type-checking, there are some problems with code that assumes that you can check constants earlier. Associated consts that are not in inherent impls cause ICEs if you try to use them in array sizes or match ranges. For similar reasons, `check_static_recursion` doesn't check them properly, so the stack goes ka-blooey if you use an associated constant that's recursively defined. That's a bit trickier to solve; I'm not entirely sure what the best approach is yet.
- Dealing with consts associated with type parameters will raise some new issues (e.g. if you have a `T: Int` type parameter and want to use `<T>::ZERO`). See rust-lang/rfcs#865.
- ~~Unused associated consts don't seem to trigger the `dead_code` lint when they should. Probably easy to fix.~~
Also, this is the first time I've been spelunking in rustc to such a large extent, so I've probably done some silly things in a couple of places.
Rather than storing the relations between free-regions in a global
table, introduce a `FreeRegionMap` data structure. regionck computes the
`FreeRegionMap` for each fn and stores the result into the tcx so that
borrowck can use it (this could perhaps be refactored to have borrowck
recompute the map, but it's a bid tedious to recompute due to the
interaction of closures and free fns). The main reason to do this is
because of #22779 -- using a global table was incorrect because when
validating impl method signatures, we want to use the free region
relationships from the *trait*, not the impl.
Fixes#22779.
This patch
1. renames libunicode to librustc_unicode,
2. deprecates several pieces of libunicode (see below), and
3. removes references to deprecated functions from
librustc_driver and libsyntax. This may change pretty-printed
output from these modules in cases involving wide or combining
characters used in filenames, identifiers, etc.
The following functions are marked deprecated:
1. char.width() and str.width():
--> use unicode-width crate
2. str.graphemes() and str.grapheme_indices():
--> use unicode-segmentation crate
3. str.nfd_chars(), str.nfkd_chars(), str.nfc_chars(), str.nfkc_chars(),
char.compose(), char.decompose_canonical(), char.decompose_compatible(),
char.canonical_combining_class():
--> use unicode-normalization crate
This makes it illegal to have unconstrained lifetimes that appear in an associated type definition. Arguably, we should prohibit all unconstrained lifetimes -- but it would break various macros. It'd be good to evaluate how large a break change it would be. But this seems like the minimal change we need to do to establish soundness, so we should land it regardless. Another variant would be to prohibit all lifetimes that appear in any impl item, not just associated types. I don't think that's necessary for soundness -- associated types are different because they can be projected -- but it would feel a bit more consistent and "obviously" safe. I'll experiment with that in the meantime.
r? @aturon
Fixes#22077.
