Pre-step towards issue #12624 and others: Introduce ExprUseVisitor, remove the
moves computation. ExprUseVisitor is a visitor that walks the AST for a
function and calls a delegate to inform it where borrows, copies, and moves
occur.
In this patch, I rewrite the gather_loans visitor to use ExprUseVisitor, but in
future patches, I think we could rewrite regionck, check_loans, and possibly
other passes to use it as well. This would refactor the repeated code between
those places that tries to determine where copies/moves/etc occur.
r? @alexcrichton
Hi rust enthusiasts,
With this patch I propose to add a "streaming" API to the existing json parser in libserialize.
By "streaming" I mean a parser that let you act on JsonEvents that are generated as while parsing happens, as opposed to parsing the entire source, generating a big data structure and working with this data structure. I think both approaches have their pros and cons so this pull request adds the streaming API, preserving the existing one.
The streaming API is simple: It consist into an Iterator<JsonEvent> that consumes an Iterator<char>. JsonEvent is an enum with values such as NumberValue(f64), BeginList, EndList, BeginObject, etc.
The user would ideally use the API as follows:
```
for evt in StreamingParser::new(src) {
match evt {
BeginList => {
// ...
}
// ...
}
}
```
The iterator provides a stack() method returning a slice of StackNodes which represent "where we currently are" in the logical structure of the json stream (for instance at "foo.bar[3].x" you get [ Key("foo"), Key("bar"), Index(3), Key("x") ].)
I wrote "ideally" above because the current way rust expands for loops, you can't call the stack() method because the iterator is already borrowed. So for know you need to manually advance the iterator in the loop. I hope this is something we can cope with, until for loops are better integrated with the compiler.
Streaming parsers are useful when you want to read from a json stream, generate a custom data structure and you know how the json is going to be structured. For example, imagine you have to parse a 3D mesh file represented in the json format. In this case you probably expect to have large arrays of vertices and using the generic parser will be very inefficient because it will create a big list of all these vertices, which you will copy into a contiguous array afterwards (so you end up doing a lot of small allocations, parsing the json once and parsing the data structure afterwards). With a streaming parser, you can add the vertices to a contiguous array as they come in without paying the cost of creating the intermediate Json data structure. You have much fewer allocations since you write directly in the final data structure and you can be smart in how you will pre-allocate it.
I added added this directly into serialize::json rather than in its own library because it turns out I can reuse most of the existing code whereas maintaining a separate library (which I did originally) forces me to duplicate this code.
I wrote this trying to minimize the size of the patch so there may be places where the code could be nicer at the expenses of more changes (let me know what you prefer).
This is my first (potential) contribution to rust, so please let me know if I am doing something wrong (maybe I should have first introduced this proposition in the mailing list, or opened a github issue, etc.?). I work a few meters away from @pknfelix so I am not too hard to find :)
When a syntax extension is loaded by the compiler, the dylib that is opened may
have other dylibs that it depends on. The dynamic linker must be able to find
these libraries on the system or else the library will fail to load.
Currently, unix gets by with the use of rpaths. This relies on the dylib not
moving around too drastically relative to its dependencies. For windows,
however, this is no rpath available, and in theory unix should work without
rpaths as well.
This modifies the compiler to add all -L search directories to the dynamic
linker's set of load paths. This is currently managed through environment
variables for each platform.
Closes#13848
When a syntax extension is loaded by the compiler, the dylib that is opened may
have other dylibs that it depends on. The dynamic linker must be able to find
these libraries on the system or else the library will fail to load.
Currently, unix gets by with the use of rpaths. This relies on the dylib not
moving around too drastically relative to its dependencies. For windows,
however, this is no rpath available, and in theory unix should work without
rpaths as well.
This modifies the compiler to add all -L search directories to the dynamic
linker's set of load paths. This is currently managed through environment
variables for each platform.
Closes#13848
The compiler has previously been producing binaries on the order of 1.8MB for
hello world programs "fn main() {}". This is largely a result of the compilation
model used by compiling entire libraries into a single object file and because
static linking is favored by default.
When linking, linkers will pull in the entire contents of an object file if any
symbol from the object file is used. This means that if any symbol from a rust
library is used, the entire library is pulled in unconditionally, regardless of
whether the library is used or not.
Traditional C/C++ projects do not normally encounter these large executable
problems because their archives (rust's rlibs) are composed of many objects.
Because of this, linkers can eliminate entire objects from being in the final
executable. With rustc, however, the linker does not have the opportunity to
leave out entire object files.
In order to get similar benefits from dead code stripping at link time, this
commit enables the -ffunction-sections and -fdata-sections flags in LLVM, as
well as passing --gc-sections to the linker *by default*. This means that each
function and each global will be placed into its own section, allowing the
linker to GC all unused functions and data symbols.
By enabling these flags, rust is able to generate much smaller binaries default.
On linux, a hello world binary went from 1.8MB to 597K (a 67% reduction in
size). The output size of dynamic libraries remained constant, but the output
size of rlibs increased, as seen below:
libarena - 2.27% bigger
libcollections - 0.64% bigger
libflate - 0.85% bigger
libfourcc - 14.67% bigger
libgetopts - 4.52% bigger
libglob - 2.74% bigger
libgreen - 9.68% bigger
libhexfloat - 13.68% bigger
liblibc - 10.79% bigger
liblog - 10.95% bigger
libnative - 8.34% bigger
libnum - 2.31% bigger
librand - 1.71% bigger
libregex - 6.43% bigger
librustc - 4.21% bigger
librustdoc - 8.98% bigger
librustuv - 4.11% bigger
libsemver - 2.68% bigger
libserialize - 1.92% bigger
libstd - 3.59% bigger
libsync - 3.96% bigger
libsyntax - 4.96% bigger
libterm - 13.96% bigger
libtest - 6.03% bigger
libtime - 2.86% bigger
liburl - 6.59% bigger
libuuid - 4.70% bigger
libworkcache - 8.44% bigger
This increase in size is a result of encoding many more section names into each
object file (rlib). These increases are moderate enough that this change seems
worthwhile to me, due to the drastic improvements seen in the final artifacts.
The overall increase of the stage2 target folder (not the size of an install)
went from 337MB to 348MB (3% increase).
Additionally, linking is generally slower when executed with all these new
sections plus the --gc-sections flag. The stage0 compiler takes 1.4s to link the
`rustc` binary, where the stage1 compiler takes 1.9s to link the binary. Three
megabytes are shaved off the binary. I found this increase in link time to be
acceptable relative to the benefits of code size gained.
This commit only enables --gc-sections for *executables*, not dynamic libraries.
LLVM does all the heavy lifting when producing an object file for a dynamic
library, so there is little else for the linker to do (remember that we only
have one object file).
I conducted similar experiments by putting a *module's* functions and data
symbols into its own section (granularity moved to a module level instead of a
function/static level). The size benefits of a hello world were seen to be on
the order of 400K rather than 1.2MB. It seemed that enough benefit was gained
using ffunction-sections that this route was less desirable, despite the lesser
increases in binary rlib size.
The compiler has previously been producing binaries on the order of 1.8MB for
hello world programs "fn main() {}". This is largely a result of the compilation
model used by compiling entire libraries into a single object file and because
static linking is favored by default.
When linking, linkers will pull in the entire contents of an object file if any
symbol from the object file is used. This means that if any symbol from a rust
library is used, the entire library is pulled in unconditionally, regardless of
whether the library is used or not.
Traditional C/C++ projects do not normally encounter these large executable
problems because their archives (rust's rlibs) are composed of many objects.
Because of this, linkers can eliminate entire objects from being in the final
executable. With rustc, however, the linker does not have the opportunity to
leave out entire object files.
In order to get similar benefits from dead code stripping at link time, this
commit enables the -ffunction-sections and -fdata-sections flags in LLVM, as
well as passing --gc-sections to the linker *by default*. This means that each
function and each global will be placed into its own section, allowing the
linker to GC all unused functions and data symbols.
By enabling these flags, rust is able to generate much smaller binaries default.
On linux, a hello world binary went from 1.8MB to 597K (a 67% reduction in
size). The output size of dynamic libraries remained constant, but the output
size of rlibs increased, as seen below:
libarena - 2.27% bigger ( 292872 => 299508)
libcollections - 0.64% bigger ( 6765884 => 6809076)
libflate - 0.83% bigger ( 186516 => 188060)
libfourcc - 14.71% bigger ( 307290 => 352498)
libgetopts - 4.42% bigger ( 761468 => 795102)
libglob - 2.73% bigger ( 899932 => 924542)
libgreen - 9.63% bigger ( 1281718 => 1405124)
libhexfloat - 13.88% bigger ( 333738 => 380060)
liblibc - 10.79% bigger ( 551280 => 610736)
liblog - 10.93% bigger ( 218208 => 242060)
libnative - 8.26% bigger ( 1362096 => 1474658)
libnum - 2.34% bigger ( 2583400 => 2643916)
librand - 1.72% bigger ( 1608684 => 1636394)
libregex - 6.50% bigger ( 1747768 => 1861398)
librustc - 4.21% bigger (151820192 => 158218924)
librustdoc - 8.96% bigger ( 13142604 => 14320544)
librustuv - 4.13% bigger ( 4366896 => 4547304)
libsemver - 2.66% bigger ( 396166 => 406686)
libserialize - 1.91% bigger ( 6878396 => 7009822)
libstd - 3.59% bigger ( 39485286 => 40902218)
libsync - 3.95% bigger ( 1386390 => 1441204)
libsyntax - 4.96% bigger ( 35757202 => 37530798)
libterm - 13.99% bigger ( 924580 => 1053902)
libtest - 6.04% bigger ( 2455720 => 2604092)
libtime - 2.84% bigger ( 1075708 => 1106242)
liburl - 6.53% bigger ( 590458 => 629004)
libuuid - 4.63% bigger ( 326350 => 341466)
libworkcache - 8.45% bigger ( 1230702 => 1334750)
This increase in size is a result of encoding many more section names into each
object file (rlib). These increases are moderate enough that this change seems
worthwhile to me, due to the drastic improvements seen in the final artifacts.
The overall increase of the stage2 target folder (not the size of an install)
went from 337MB to 348MB (3% increase).
Additionally, linking is generally slower when executed with all these new
sections plus the --gc-sections flag. The stage0 compiler takes 1.4s to link the
`rustc` binary, where the stage1 compiler takes 1.9s to link the binary. Three
megabytes are shaved off the binary. I found this increase in link time to be
acceptable relative to the benefits of code size gained.
This commit only enables --gc-sections for *executables*, not dynamic libraries.
LLVM does all the heavy lifting when producing an object file for a dynamic
library, so there is little else for the linker to do (remember that we only
have one object file).
I conducted similar experiments by putting a *module's* functions and data
symbols into its own section (granularity moved to a module level instead of a
function/static level). The size benefits of a hello world were seen to be on
the order of 400K rather than 1.2MB. It seemed that enough benefit was gained
using ffunction-sections that this route was less desirable, despite the lesser
increases in binary rlib size.
Similar to my recent changes to ~[T]/&[T], these changes remove the vstore abstraction and represent str types as ~(str) and &(str). The Option<uint> in ty_str is the length of the string, None if the string is dynamically sized.
This addresses the ICE from #13763, but it does not allow the test to compile,
due to #13768. An alternate test was checked in in the meantime.
Closes#13763
This addresses the ICE from #13763, but it does not allow the test to compile,
due to #13768. An alternate test was checked in in the meantime.
Closes#13763
It didn't work because it tried to call itself but symbols are not
exported as default in executables.
Note that `fun5` is not internal anymore since it is in library.
Second commit removes/updates some old tests.
It didn't work because it tried to call itself but symbols are not
exported as default in executables.
Note that `fun5` is not internal anymore since it is in library.
Implements [RFC 7](https://github.com/rust-lang/rfcs/blob/master/active/0007-regexps.md) and will hopefully resolve#3591. The crate is marked as experimental. It includes a syntax extension for compiling regexps to native Rust code.
Embeds and passes the `basic`, `nullsubexpr` and `repetition` tests from [Glenn Fowler's (slightly modified by Russ Cox for leftmost-first semantics) testregex test suite](http://www2.research.att.com/~astopen/testregex/testregex.html). I've also hand written a plethora of other tests that exercise Unicode support, the parser, public API, etc. Also includes a `regex-dna` benchmark for the shootout.
I know the addition looks huge at first, but consider these things:
1. More than half the number of lines is dedicated to Unicode character classes.
2. Of the ~4,500 lines remaining, 1,225 of them are comments.
3. Another ~800 are tests.
4. That leaves 2500 lines for the meat. The parser is ~850 of them. The public API, compiler, dynamic VM and code generator (for `regexp!`) make up the rest.
moves computation. ExprUseVisitor is a visitor that walks the AST for a
function and calls a delegate to inform it where borrows, copies, and moves
occur.
In this patch, I rewrite the gather_loans visitor to use ExprUseVisitor, but in
future patches, I think we could rewrite regionck, check_loans, and possibly
other passes to use it as well. This would refactor the repeated code between
those places that tries to determine where copies/moves/etc occur.
Specifically, the method parameter cardinality mismatch or missing
method error message span now gets method itself exactly. It was the
whole expression.
Closes#9390Closes#13684Closes#13709
This patch removes the special auto-rooting for `@` from the borrow checker. With `@` moving into a library, it doesn't make sense to keep this code around anymore. It also simplifies `trans` by removing root checking from there
@nikomatsakis
Closes: #11586
This is really only useful for #[cfg()]. For example:
```rust
enum Foo {
Bar,
Baz,
#[cfg(blob)]
Blob
}
fn match_foos(f: &Foo) {
match *f {
Bar => {}
Baz => {}
#[cfg(blob)]
Blob => {}
}
}
```
This is a kind of weird place to allow attributes, so it should probably
be discussed before merging.
The constructor for `TaskBuilder` is being changed to an associated
function called `new` for consistency with the rest of the standard
library.
Closes#13666
[breaking-change]
The constructor for `TaskBuilder` is being changed to an associated
function called `new` for consistency with the rest of the standard
library.
Closes#13666
[breaking-change]
Specifically, the method parameter cardinality mismatch or missing
method error message span now gets method itself exactly. It was the
whole expression.
Closes#9390Closes#13684Closes#13709
This allows the use of syntax extensions when cross-compiling (fixing #12102). It does this by encoding the target triple in the crate metadata and checking it when searching for files. Currently the crate triple must match the host triple when there is a macro_registrar_fn, it must match the target triple when linking, and can match either when only macro_rules! macros are used.
due to carelessness, this is pretty much a duplicate of https://github.com/mozilla/rust/pull/13450.
This adds the target triple to the crate metadata.
When searching for a crate the phase (link, syntax) is taken into account.
During link phase only crates matching the target triple are considered.
During syntax phase, either the target or host triple will be accepted, unless
the crate defines a macro_registrar, in which case only the host triple will
match.
This alters the borrow checker's requirements on invoking closures from
requiring an immutable borrow to requiring a unique immutable borrow. This means
that it is illegal to invoke a closure through a `&` pointer because there is no
guarantee that is not aliased. This does not mean that a closure is required to
be in a mutable location, but rather a location which can be proven to be
unique (often through a mutable pointer).
For example, the following code is unsound and is no longer allowed:
type Fn<'a> = ||:'a;
fn call(f: |Fn|) {
f(|| {
f(|| {})
});
}
fn main() {
call(|a| {
a();
});
}
There is no replacement for this pattern. For all closures which are stored in
structures, it was previously allowed to invoke the closure through `&self` but
it now requires invocation through `&mut self`.
The standard library has a good number of violations of this new rule, but the
fixes will be separated into multiple breaking change commits.
Closes#12224
The filestem of the desired output isn't necessarily a valid crate id, and
calling unwrap() will trigger an ICE in rustc. This tries a little harder to
infer a "valid crate id" from a crate, with an eventual fallback to a generic
crate id if alll else fails.
Closes#11107
This alters the borrow checker's requirements on invoking closures from
requiring an immutable borrow to requiring a unique immutable borrow. This means
that it is illegal to invoke a closure through a `&` pointer because there is no
guarantee that is not aliased. This does not mean that a closure is required to
be in a mutable location, but rather a location which can be proven to be
unique (often through a mutable pointer).
For example, the following code is unsound and is no longer allowed:
type Fn<'a> = ||:'a;
fn call(f: |Fn|) {
f(|| {
f(|| {})
});
}
fn main() {
call(|a| {
a();
});
}
There is no replacement for this pattern. For all closures which are stored in
structures, it was previously allowed to invoke the closure through `&self` but
it now requires invocation through `&mut self`.
The standard library has a good number of violations of this new rule, but the
fixes will be separated into multiple breaking change commits.
Closes#12224
[breaking-change]
The BSD builders are failing with a different error that is not a timeout error
(Connection reset by peer), so this test isn't really all that useful on
freebsd. Due to a lack of a better idea of how to test a connect timeout, this
test is going to just be ignored for now.
The BSD builders are failing with a different error that is not a timeout error
(Connection reset by peer), so this test isn't really all that useful on
freebsd. Due to a lack of a better idea of how to test a connect timeout, this
test is going to just be ignored for now.
Now with proper checking of enums and allows unsized fields as the last field in a struct or variant. This PR only checks passing of unsized types and distinguishing them from sized ones. To be safe we also need to control storage.
Closes issues #12969 and #13121, supersedes #13375 (all the discussion there is valid here too).
