The existing APIs for spawning processes took strings for the command
and arguments, but the underlying system may not impose utf8 encoding,
so this is overly limiting.
The assumption we actually want to make is just that the command and
arguments are viewable as [u8] slices with no interior NULLs, i.e., as
CStrings. The ToCStr trait is a handy bound for types that meet this
requirement (such as &str and Path).
However, since the commands and arguments are often a mixture of
strings and paths, it would be inconvenient to take a slice with a
single T: ToCStr bound. So this patch revamps the process creation API
to instead use a builder-style interface, called `Command`, allowing
arguments to be added one at a time with differing ToCStr
implementations for each.
The initial cut of the builder API has some drawbacks that can be
addressed once issue #13851 (libstd as a facade) is closed. These are
detailed as FIXMEs.
Closes#11650.
[breaking-change]
There's no need to include this specific flag just for android. We can
already deal with what it tries to solve by using -C linker=/path/to/cc
and -C ar=/path/to/ar. The Makefiles for rustc already set this up when
we're crosscompiling.
I did add the flag to compiletest though so it can find gdb. Though, I'm
pretty sure we don't run debuginfo tests on android anyways right now.
[breaking-change]
This implements set_timeout() for std::io::Process which will affect wait()
operations on the process. This follows the same pattern as the rest of the
timeouts emerging in std::io::net.
The implementation was super easy for everything except libnative on unix
(backwards from usual!), which required a good bit of signal handling. There's a
doc comment explaining the strategy in libnative. Internally, this also required
refactoring the "helper thread" implementation used by libnative to allow for an
extra helper thread (not just the timer).
This is a breaking change in terms of the io::Process API. It is now possible
for wait() to fail, and subsequently wait_with_output(). These two functions now
return IoResult<T> due to the fact that they can time out.
Additionally, the wait_with_output() function has moved from taking `&mut self`
to taking `self`. If a timeout occurs while waiting with output, the semantics
are undesirable in almost all cases if attempting to re-wait on the process.
Equivalent functionality can still be achieved by dealing with the output
handles manually.
[breaking-change]
cc #13523
The goal of this refactoring is to make the rustc driver code easier to understand and use. Since this is as close to an API as we have, I think it is important that it is nice. On getting stuck in, I found that there wasn't as much to change as I'd hoped to make the stage... functions easier to use by tools (which is a good thing :-) ).
This patch only moves code around - mostly just moving code to different files, but a few extracted method refactorings too. To summarise the changes: I added driver::config which handles everything about configuring the compiler. driver::session now just defines and builds session objects. I moved driver code from librustc/lib.rs to librustc/driver/mod.rs so all the code is one place. I extracted methods to make emulating the compiler without being the compiler a little easier. Within the driver directory, I moved code around to more logically fit in the modules.
This commit revisits the `cast` module in libcore and libstd, and scrutinizes
all functions inside of it. The result was to remove the `cast` module entirely,
folding all functionality into the `mem` module. Specifically, this is the fate
of each function in the `cast` module.
* transmute - This function was moved to `mem`, but it is now marked as
#[unstable]. This is due to planned changes to the `transmute`
function and how it can be invoked (see the #[unstable] comment).
For more information, see RFC 5 and #12898
* transmute_copy - This function was moved to `mem`, with clarification that is
is not an error to invoke it with T/U that are different
sizes, but rather that it is strongly discouraged. This
function is now #[stable]
* forget - This function was moved to `mem` and marked #[stable]
* bump_box_refcount - This function was removed due to the deprecation of
managed boxes as well as its questionable utility.
* transmute_mut - This function was previously deprecated, and removed as part
of this commit.
* transmute_mut_unsafe - This function doesn't serve much of a purpose when it
can be achieved with an `as` in safe code, so it was
removed.
* transmute_lifetime - This function was removed because it is likely a strong
indication that code is incorrect in the first place.
* transmute_mut_lifetime - This function was removed for the same reasons as
`transmute_lifetime`
* copy_lifetime - This function was moved to `mem`, but it is marked
`#[unstable]` now due to the likelihood of being removed in
the future if it is found to not be very useful.
* copy_mut_lifetime - This function was also moved to `mem`, but had the same
treatment as `copy_lifetime`.
* copy_lifetime_vec - This function was removed because it is not used today,
and its existence is not necessary with DST
(copy_lifetime will suffice).
In summary, the cast module was stripped down to these functions, and then the
functions were moved to the `mem` module.
transmute - #[unstable]
transmute_copy - #[stable]
forget - #[stable]
copy_lifetime - #[unstable]
copy_mut_lifetime - #[unstable]
[breaking-change]
The goal of this refactoring is to make the rustc driver code easier to understand and use. Since this is as close to an API as we have, I think it is important that it is nice. On getting stuck in, I found that there wasn't as much to change as I'd hoped to make the stage... fns easier to use by tools.
This patch only moves code around - mostly just moving code to different files, but a few extracted method refactorings too. To summarise the changes: I added driver::config which handles everything about configuring the compiler. driver::session now just defines and builds session objects. I moved driver code from librustc/lib.rs to librustc/driver/mod.rs so all the code is one place. I extracted methods to make emulating the compiler without being the compiler a little easier. Within the driver directory, I moved code around to more logically fit in the modules.
Currently, rustc requires that a linkage be a product of 100% rlibs or 100%
dylibs. This is to satisfy the requirement that each object appear at most once
in the final output products. This is a bit limiting, and the upcoming libcore
library cannot exist as a dylib, so these rules must change.
The goal of this commit is to enable *some* use cases for mixing rlibs and
dylibs, primarily libcore's use case. It is not targeted at allowing an
exhaustive number of linkage flavors.
There is a new dependency_format module in rustc which calculates what format
each upstream library should be linked as in each output type of the current
unit of compilation. The module itself contains many gory details about what's
going on here.
cc #10729
Pre-step towards issue #12624 and others: Introduce ExprUseVisitor, remove the
moves computation. ExprUseVisitor is a visitor that walks the AST for a
function and calls a delegate to inform it where borrows, copies, and moves
occur.
In this patch, I rewrite the gather_loans visitor to use ExprUseVisitor, but in
future patches, I think we could rewrite regionck, check_loans, and possibly
other passes to use it as well. This would refactor the repeated code between
those places that tries to determine where copies/moves/etc occur.
r? @alexcrichton
The compiler has previously been producing binaries on the order of 1.8MB for
hello world programs "fn main() {}". This is largely a result of the compilation
model used by compiling entire libraries into a single object file and because
static linking is favored by default.
When linking, linkers will pull in the entire contents of an object file if any
symbol from the object file is used. This means that if any symbol from a rust
library is used, the entire library is pulled in unconditionally, regardless of
whether the library is used or not.
Traditional C/C++ projects do not normally encounter these large executable
problems because their archives (rust's rlibs) are composed of many objects.
Because of this, linkers can eliminate entire objects from being in the final
executable. With rustc, however, the linker does not have the opportunity to
leave out entire object files.
In order to get similar benefits from dead code stripping at link time, this
commit enables the -ffunction-sections and -fdata-sections flags in LLVM, as
well as passing --gc-sections to the linker *by default*. This means that each
function and each global will be placed into its own section, allowing the
linker to GC all unused functions and data symbols.
By enabling these flags, rust is able to generate much smaller binaries default.
On linux, a hello world binary went from 1.8MB to 597K (a 67% reduction in
size). The output size of dynamic libraries remained constant, but the output
size of rlibs increased, as seen below:
libarena - 2.27% bigger
libcollections - 0.64% bigger
libflate - 0.85% bigger
libfourcc - 14.67% bigger
libgetopts - 4.52% bigger
libglob - 2.74% bigger
libgreen - 9.68% bigger
libhexfloat - 13.68% bigger
liblibc - 10.79% bigger
liblog - 10.95% bigger
libnative - 8.34% bigger
libnum - 2.31% bigger
librand - 1.71% bigger
libregex - 6.43% bigger
librustc - 4.21% bigger
librustdoc - 8.98% bigger
librustuv - 4.11% bigger
libsemver - 2.68% bigger
libserialize - 1.92% bigger
libstd - 3.59% bigger
libsync - 3.96% bigger
libsyntax - 4.96% bigger
libterm - 13.96% bigger
libtest - 6.03% bigger
libtime - 2.86% bigger
liburl - 6.59% bigger
libuuid - 4.70% bigger
libworkcache - 8.44% bigger
This increase in size is a result of encoding many more section names into each
object file (rlib). These increases are moderate enough that this change seems
worthwhile to me, due to the drastic improvements seen in the final artifacts.
The overall increase of the stage2 target folder (not the size of an install)
went from 337MB to 348MB (3% increase).
Additionally, linking is generally slower when executed with all these new
sections plus the --gc-sections flag. The stage0 compiler takes 1.4s to link the
`rustc` binary, where the stage1 compiler takes 1.9s to link the binary. Three
megabytes are shaved off the binary. I found this increase in link time to be
acceptable relative to the benefits of code size gained.
This commit only enables --gc-sections for *executables*, not dynamic libraries.
LLVM does all the heavy lifting when producing an object file for a dynamic
library, so there is little else for the linker to do (remember that we only
have one object file).
I conducted similar experiments by putting a *module's* functions and data
symbols into its own section (granularity moved to a module level instead of a
function/static level). The size benefits of a hello world were seen to be on
the order of 400K rather than 1.2MB. It seemed that enough benefit was gained
using ffunction-sections that this route was less desirable, despite the lesser
increases in binary rlib size.
The compiler has previously been producing binaries on the order of 1.8MB for
hello world programs "fn main() {}". This is largely a result of the compilation
model used by compiling entire libraries into a single object file and because
static linking is favored by default.
When linking, linkers will pull in the entire contents of an object file if any
symbol from the object file is used. This means that if any symbol from a rust
library is used, the entire library is pulled in unconditionally, regardless of
whether the library is used or not.
Traditional C/C++ projects do not normally encounter these large executable
problems because their archives (rust's rlibs) are composed of many objects.
Because of this, linkers can eliminate entire objects from being in the final
executable. With rustc, however, the linker does not have the opportunity to
leave out entire object files.
In order to get similar benefits from dead code stripping at link time, this
commit enables the -ffunction-sections and -fdata-sections flags in LLVM, as
well as passing --gc-sections to the linker *by default*. This means that each
function and each global will be placed into its own section, allowing the
linker to GC all unused functions and data symbols.
By enabling these flags, rust is able to generate much smaller binaries default.
On linux, a hello world binary went from 1.8MB to 597K (a 67% reduction in
size). The output size of dynamic libraries remained constant, but the output
size of rlibs increased, as seen below:
libarena - 2.27% bigger ( 292872 => 299508)
libcollections - 0.64% bigger ( 6765884 => 6809076)
libflate - 0.83% bigger ( 186516 => 188060)
libfourcc - 14.71% bigger ( 307290 => 352498)
libgetopts - 4.42% bigger ( 761468 => 795102)
libglob - 2.73% bigger ( 899932 => 924542)
libgreen - 9.63% bigger ( 1281718 => 1405124)
libhexfloat - 13.88% bigger ( 333738 => 380060)
liblibc - 10.79% bigger ( 551280 => 610736)
liblog - 10.93% bigger ( 218208 => 242060)
libnative - 8.26% bigger ( 1362096 => 1474658)
libnum - 2.34% bigger ( 2583400 => 2643916)
librand - 1.72% bigger ( 1608684 => 1636394)
libregex - 6.50% bigger ( 1747768 => 1861398)
librustc - 4.21% bigger (151820192 => 158218924)
librustdoc - 8.96% bigger ( 13142604 => 14320544)
librustuv - 4.13% bigger ( 4366896 => 4547304)
libsemver - 2.66% bigger ( 396166 => 406686)
libserialize - 1.91% bigger ( 6878396 => 7009822)
libstd - 3.59% bigger ( 39485286 => 40902218)
libsync - 3.95% bigger ( 1386390 => 1441204)
libsyntax - 4.96% bigger ( 35757202 => 37530798)
libterm - 13.99% bigger ( 924580 => 1053902)
libtest - 6.04% bigger ( 2455720 => 2604092)
libtime - 2.84% bigger ( 1075708 => 1106242)
liburl - 6.53% bigger ( 590458 => 629004)
libuuid - 4.63% bigger ( 326350 => 341466)
libworkcache - 8.45% bigger ( 1230702 => 1334750)
This increase in size is a result of encoding many more section names into each
object file (rlib). These increases are moderate enough that this change seems
worthwhile to me, due to the drastic improvements seen in the final artifacts.
The overall increase of the stage2 target folder (not the size of an install)
went from 337MB to 348MB (3% increase).
Additionally, linking is generally slower when executed with all these new
sections plus the --gc-sections flag. The stage0 compiler takes 1.4s to link the
`rustc` binary, where the stage1 compiler takes 1.9s to link the binary. Three
megabytes are shaved off the binary. I found this increase in link time to be
acceptable relative to the benefits of code size gained.
This commit only enables --gc-sections for *executables*, not dynamic libraries.
LLVM does all the heavy lifting when producing an object file for a dynamic
library, so there is little else for the linker to do (remember that we only
have one object file).
I conducted similar experiments by putting a *module's* functions and data
symbols into its own section (granularity moved to a module level instead of a
function/static level). The size benefits of a hello world were seen to be on
the order of 400K rather than 1.2MB. It seemed that enough benefit was gained
using ffunction-sections that this route was less desirable, despite the lesser
increases in binary rlib size.
This flag to the linker asks it to strip away all dead code during linking, as
well as dead data. This reduces the size of hello world from 1.7MB to 458K on my
system (70% reduction).
I have not seen this impact link times negatively, and I have seen this pass
'make check' successfully. I am slightly wary of adding this option, but the
benefits are so huge tha I think we should work hard to work around any issues
rather than avoid using the flag entirely.
This adds the target triple to the crate metadata.
When searching for a crate the phase (link, syntax) is taken into account.
During link phase only crates matching the target triple are considered.
During syntax phase, either the target or host triple will be accepted, unless
the crate defines a macro_registrar, in which case only the host triple will
match.
The filestem of the desired output isn't necessarily a valid crate id, and
calling unwrap() will trigger an ICE in rustc. This tries a little harder to
infer a "valid crate id" from a crate, with an eventual fallback to a generic
crate id if alll else fails.
Closes#11107
Many of the instances of setting a global error variable ended up leaving a
dangling pointer into free'd memory. This changes the method of error
transmission to strdup any error and "relinquish ownership" to rustc when it
gets an error. The corresponding Rust code will then free the error as
necessary.
Closes#12865
This is a bit of an interesting upgrade to LLVM. Upstream LLVM has started using C++11 features, so they require a C++11 compiler to build. I've updated all the bots to have a C++11 compiler, and they appear to be building LLVM successfully:
* Linux bots - I added gcc/g++ 4.7 (good enough)
* Android bots - same as the linux ones
* Mac bots - I installed the most recent command line tools for Lion which gives us clang 3.2, but LLVM wouldn't build unless it was explicitly asked to link to `libc++` instead of `libstdc++`. This involved tweaking `mklldeps.py` and the `configure` script to get things to work out
* Windows bots - mingw-w64 has gcc 4.8.1 which is sufficient for building LLVM (hurray!)
* BSD bots - I updated FreeBSD to 10.0 which brought with it a relevant version of clang.
The largest fallout I've seen so far is that the test suite doesn't work at all on FreeBSD 10. We've already stopped gating on FreeBSD due to #13427 (we used to be on freebsd 9), so I don't think this puts us in too bad of a situation. I will continue to attempt to fix FreeBSD and the breakage on there.
The LLVM update brings with it all of the recently upstreamed LLVM patches. We only have one local patch now which is just an optimization, and isn't required to use upstream LLVM. I want to maintain compatibility with LLVM 3.3 and 3.4 while we can, and this upgrade is keeping us up to date with the 3.5 release. Once 3.5 is release we will in theory no longer require a bundled LLVM.
If a linker finds both a static and a dynamic version of the same library, then
the linker often chooses the dynamic version. This is surprising when a native
library is specified as being "static" in rust source. This modifies the linker
command line to obey the hints given in rust source files and instructing the
linker to prefer a particular version of a found library.
Unfortunately, this patch has no effect on osx because the linker supports
no such hint, and it also has no effect on windows because the linker apparently
just ignores it. For now this is predominately used to enable the previous patch
of linking to libstdc++ statically, but more support would need to be added for
this in the future if we wanted to officially support it.
cc #12557 (doesn't close because it doesn't support OSX and windows)
Previously, upstream C libraries were linked in a nondeterministic fashion
because they were collected through iter_crate_data() which is a nodeterministic
traversal of a hash map. When upstream rlibs had interdependencies among their
native libraries (such as libfoo depending on libc), then the ordering would
occasionally be wrong, causing linkage to fail.
This uses the topologically sorted list of libraries to collect native
libraries, so if a native library depends on libc it just needs to make sure
that the rust crate depends on liblibc.
libstd: Implement `StrBuf`, a new string buffer type like `Vec`, and port all code over to use it.
Rebased & tests-fixed version of https://github.com/mozilla/rust/pull/13269
When calculating the sysroot, it's more accurate to use realpath() rather than
just one readlink() to account for any intermediate symlinks that the rustc
binary resolves itself to.
For rpath, realpath() is necessary because the rpath must dictate a relative
rpath from the destination back to the originally linked library, which works
more robustly if there are no symlinks involved.
Concretely, any binary generated on OSX into $TMPDIR requires an absolute rpath
because the temporary directory is behind a symlink with one layer of
indirection. This symlink causes all relative rpaths to fail to resolve.
cc #11734
cc #11857
Concerns have been raised about using absolute rpaths in #11746, and this is the
first step towards not relying on rpaths at all. The only current use case for
an absolute rpath is when a non-installed rust builds an executable that then
moves from is built location. The relative rpath back to libstd and absolute
rpath to the installation directory still remain (CFG_PREFIX).
Closes#11746
Rebasing of #12754
`Reader`, `Writer`, `MemReader`, `MemWriter`, and `MultiWriter` now work with `Vec<u8>` instead of `~[u8]`. This does introduce some extra copies since `from_utf8_owned` isn't usable anymore, but I think that can't be helped until `~str`'s representation changes.
Rust currently defaults to `RelocPIC` regardless. This patch adds a new
codegen option that allows choosing different relocation-model. The
available models are:
- default (Use the target-specific default model)
- static
- pic
- no-pic
For a more detailed information use `llc --help`
Rust currently defaults to `RelocPIC` regardless. This patch adds a new
codegen option that allows choosing different relocation-model. The
available models are:
- default (Use the target-specific default model)
- static
- pic
- no-pic
For a more detailed information use `llc --help`
A variety of stuff here, mostly aimed at making `make install` work correctly with `--libdir` and `--mandir`. `make install` again goes through `install.sh`.
