This increases the maximum supported file size (previously limited to 4GB)
and avoids potential issues with u32 to u64 conversions, which are no longer
needed.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Add batch flag to remote-test-server
When using this flag, the stdout and stderr are sent in a single batch instead of being streamed. It also used `Command::output` instead of `Command::spawn`. This is useful for targets that might support std but not threading (Eg: UEFI).
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
When using this flag, the stdout and stderr is sent in a single batch
instead of being streamed. It also used `Command::output` instead of
`Command::spawn`. This is useful for targets that might support std
but not threading (Eg: UEFI).
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
This allows retrying binding TCP Socket multiple times. This is useful
when using emulators as network might not be available in the beginning.
This was orignally implemented in https://github.com/rust-lang/rust/pull/100316
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
The /data/tmp directory does not exist, at least not on recent versions
of Android, which currently leads to test failures on that platform. I
checked a virtual device running AOSP master and a Nexus 5 running
Android Marshmallow and on both devices the /data/tmp directory does
not exist and /data/local/tmp does, so let's switch to /data/local/tmp.
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
Some tests (e.g. ui-fulldeps/create-dir-all-bare.rs) assume that
RUST_TEST_TMPDIR exists on the system running the test. Expand
remote-test-{server,client} such that a tmp directory is created on the
remote runner and this environment variable will point at it.
`compiletest` and `remote-test-client`:
The command line for `remote-test-client` was changed slightly
to allow cross-platform compatible paths. The old way of supplying
the support libs was by joining their paths with the executable path
with `:`. This caused Windows-style paths to be split after the
directory letter. Now, the number of support libs is provided
as a parameter as well, and the support lib paths are split off
from the regular args in the client.
`remote-test-server`:
- Marked Unix-only parts as such and implemented Windows alternatives
- On Windows `LD_LIBRARY_PATH` doesn't exist. Libraries are
loaded from `PATH` though, so that's the way around it.
- Tiny cleanup: `Command::args`/`envs` instead of manually
looping over them
- The temp path for Windows has to be set via environment variable,
since there isn't a global temp directory that would work on every
machine (as a static string)
Currently our slowest test suite on android, run-pass, takes over 5 times longer
than the x86_64 component (~400 -> ~2200s). Typically QEMU emulation does indeed
add overhead, but not 5x for this kind of workload. One of the slowest parts of
the Android process is that *compilation* happens serially. Tests themselves
need to run single-threaded on the emulator (due to how the test harness works)
and this forces the compiles themselves to be single threaded.
Now Travis gives us more than one core per machine, so it'd be much better if we
could take advantage of them! The emulator itself is still fundamentally
single-threaded, but we should see a nice speedup by sending binaries for it to
run much more quickly.
It turns out that we've already got all the tools to do this in-tree. The
qemu-test-{server,client} that are in use for the ARM Linux testing are a
perfect match for the Android emulator. This commit migrates the custom adb
management code in compiletest/rustbuild to the same qemu-test-{server,client}
implementation that ARM Linux uses.
This allows us to lift the parallelism restriction on the compiletest test
suites, namely run-pass. Consequently although we'll still basically run the
tests themselves in single threaded mode we'll be able to compile all of them in
parallel, keeping the pipeline much more full and using more cores for the work
at hand. Additionally the architecture here should be a bit speedier as it
should have less overhead than adb which is a whole new process on both the host
and the emulator!
Locally on an 8 core machine I've seen the run-pass test suite speed up from
taking nearly an hour to only taking 6 minutes. I don't think we'll see quite a
drastic speedup on Travis but I'm hoping this change can place the Android tests
well below 2 hours instead of just above 2 hours.
Because the client/server here are now repurposed for more than just QEMU,
they've been renamed to `remote-test-{server,client}`.
Note that this PR does not currently modify how debuginfo tests are executed on
Android. While parallelizable it wouldn't be quite as easy, so that's left to
another day. Thankfully that test suite is much smaller than the run-pass test
suite.
As a final fix I discovered that the ARM and Android test suites were actually
running all library unit tests (e.g. stdtest, coretest, etc) twice. I've
corrected that to only run tests once which should also give a nice boost in
overall cycle time here.
