Commit graph

14 commits

Author SHA1 Message Date
Zach Reizner
d09392e37e uprev rust-toolchain and fix clippy warnings
Some judgement calls were made about unnecessary wrapping. Usually they
would get resolved by removing the wrapping or returning a convenient
error, but the ones that returned results for consistency with other
functions were added to the allow list.

The error handling in the usb code had a lot of unit error types which
is now a clippy lint. This was resolved by either removing the result
entirely or returning a convenient error.

The field_reassign_with_default lint is faulty and was added to the list
of supressions. This affected virtio-wayland code.

BUG=b:179277332
TEST=cargo clippy with rustc 1.50+

Change-Id: Ie812cdeaf7c42f4f2b47b1dc87f05a7c87a60f8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2757510
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-22 21:41:07 +00:00
Dennis Kempin
3f63661afc Run integration_tests by calling crosvm binary
Instead of instanciating crosvm directly, we can start the binary as
a sub-process.
This includes parsing of crosvm options in the tests, and makes the
test cases closer to real-world usage.

To make make this possible, we need to make sure that the crosvm binary
is uploaded to the VM before running the test, which is done by the
sync_so script, which is baked into the builder container.
We prevent future container re-builds for just maintaining the script,
I have removed them from the container, and call the scripts from the
local source directly.

The test runner is also updated to ensure all package binaries are
built (currently only tests are built).

BUG=b:182841358
TEST=./test_all passes

Change-Id: I7dfd21abcb2b90fe125eb43f85572fbf645b888a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2744280
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-03-19 20:35:53 +00:00
Keiichi Watanabe
5dc0f60d44 kokoro: Add rust-vmm/vhost in manifest.xml
BUG=b:181227406
TEST=ci/kokoro/simulate_all

Change-Id: I65065963daa9e96457ed9a64f01e34ad93bd8667
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2762488
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-17 19:12:08 +00:00
Dennis Kempin
dcf540be92 Add SEPARATE_WORKSPACE requirement for run_tests
The io_uring and fuzz crate cannot be tested in the same workspace
via `cargo test -p io_uring -p fuzz`.
This change allows test_runner.py to run tests independently by
cd'ing into the crates directory and running cargo test in there.

BUG=b:181672910
BUG=b:181673923
TEST=Tests for the above mentioned crates can be enabled and run, but
test failures prevent them from being enabled so far.

Change-Id: Ia03868d53e508549fe2f071da399b982359a8834
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2749772
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-12 23:36:13 +00:00
Dennis Kempin
f248a8ec2b Fix Kokoro build with manifest uprev
Kokoro was failing since http://crrev.com/c/2744968 depends on recent
cras changes that were not included in the current Kokoro manifest.

BUG=None
TEST=./ci/kokoro/simulate_all passes

Change-Id: I470844af73b9f89ac8cbf9f5fcf8706d01924279
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2746014
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-10 06:16:25 +00:00
Dennis Kempin
3682372764 ci: Upgrade test-vm to debian bullseye
This change updates the test VM to debian bullseye, primarily to enable
io_uring features that are not available in the 4.16 kernel shipping
with debian buster.

We are not updating the builder containers to bullseye, as newer
versions of qemu fail to start on kokoro. See b/181359683

Since it was needed for debugging this CL, it includes a feature to
debug kokoro builders via SSH. It can be enabled by setting
DEBUG_SSH_KEY from the fusion2 UI.

BUG=b:178228512
TEST=./test_all passes

Change-Id: I22f1678577183a0aace80245a87c31907221cc09
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2721062
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-10 03:37:00 +00:00
Daniel Verkamp
0a91c96437 arch: rewrite FDT writer in native rust
This removes some unsafe code, improving the interface so that it cannot
be misused (e.g. previously, different Vec<u8> instances could be passed
to fdt functions that did not validate the contents).

The new implementation also adds some extra error checking to catch
invalid string values in all API entry points that accept strings, as
well as out-of-order node nesting that would result in DTB data that did
not conform to the spec.

BUG=b:175729255
TEST=cargo test -p arch
TEST=Boot Crostini on kevin (arm)
TEST=diff old and new dts from `dtc -I fs -O dts /proc/device-tree`

Change-Id: I567d562ed1b447aa0d282b68c71758edea13ffc0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2713569
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-03-03 01:29:15 +00:00
Dennis Kempin
4d2253b529 Kokoro: Extensive polishing and bugfixing
This CL fixes some of the issues that prevented the
test system from running on other workstations.

- Builders run by Kokoro will not use a scratch mount.
  It's not needed, and will cause issues if stored on
  /tmp with noexec set.
- Running podman with label=disable to prevent selinux
  issues.

The run_tests implementation has been moved to a separate
file and updated with parsing of cargo test output. This
allows simpler test output and integration with the
test results UI in sponge.

The sponge test UI can make it much easier to see which
tests failed, and to find the log of that test case.

This CL also includes an ./ci/kokoro/uprev script to
uprev the manifest versions. And runs the uprev.

BUG=b:174861002
TEST=Tested by forcing a kokoro build with this CL

Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-10 22:04:43 +00:00
Dennis Kempin
26cc51e588 Make Kokoro v2 happy with new config
Fixes some nits that cause kokoro builds to fail and updates
the Kokoro configs to match the new Kokoro-side configs
submitted in cl/355894530.

With this CL Kokoro v2 should be passing and can be enabled as
a presubmit.

BUG=b:179289824
TEST=Kokoro passes

Change-Id: I8c9259407bf1eb932b05376b35cb88d2e7d6058c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2678792
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-02-06 05:26:04 +00:00
Dennis Kempin
0dbb9808a6 Add Kokoro for ci/builders
Adds the crosvm-side infrastructure to build and test
in kokoro.
There is a build script for testing on x86, aarch64
and a separte script for analysis (clippy, fmt).
These will run in parallel on Kokoro. To test the
scripts locally, a simulate script is provided.

Runtime on my workstation:
- aarch64: 10m
- x86: 2:30m
- analysis: 1:40m

BUG=b:177951955
TEST=./ci/kokoro/simulate_all

Change-Id: I2f666ec768e6c3391a258dc7f0cbd999ad9b2fb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2654413
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
2021-02-05 03:01:55 +00:00
Dennis Kempin
aa0717443e Add VM to builders
The VM allows both the x86 and aarch64 builders to run tests which
require access to kernel devices that docker would prevent us
from accessing.
The VM is automatically started in the endpoint script, and the
environment is set up to execute `cargo test` binaries inside the VM.

This allows for convenient interactive usage.
The VM is also integrated into the run_tests scripts, executing tests
that do not require special privileges first, while the VM is still
booting up, then execuing the remaining tests in the VM.

BUG=b:177228167,b:177951276
TEST=These test calls pass:
./ci/builder --vm ./run_tests
./ci/aarch64_builder --vm ./run_tests
./run_tests

Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-02-05 03:01:53 +00:00
Dennis Kempin
b40ec47309 Make Kokoro happy
A few things happened:

- A recent minigbm change broke compilation for debian, so
  I am pinning the commit SHA for the Dockerfile.
- Newer rust versions come with new clippy errors, which I will
  have to look at in a separate change. For now I am pinning our
  rust version.
- Some repos have transitioned to a main branch. Fixed our uprev
  script to handle those.
- Renamed SYSROOT so we won't confuse pkg-config.

Now Kokoro is finally happy again!

BUG=None
TEST=wrapped_smoke_tests

Change-Id: I76294abe35fd84b305124158b3942a321651982d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643779
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-01-22 22:32:37 +00:00
Dennis Kempin
2118d6faa3 Test Runner for crosvm
The ./run_tests script will select which tests to run on the host.
Unfortunately a lot of our tests require dependencies or access
to certain kernel devices. We cannot run all tests on all hosts.

The test runner works with the CI builders:

To run all x86 tests:
./ci/builder ./run_tests --all

To run tests on aarch64:
./ci/aarch64_builder ./run_tests --all

Or to just run tests that can be run on a standard debian buster
system without obscure chromeos dependencies:

./run_tests

Eventually, the test runner should run tests via the VM of the
aarch64_builder, currently only those tests that can be run with
user-space emulation are executed.

Currently some tests remain disabled in the runner, as they do not
pass in the CI builders yet. These will be enabled as the builders
are set up to allow for them to be run.

BUG=b:173833661
TEST=Ran the above commands.

Change-Id: Id69027793348f4d3f20adf5333d8bdfff1aa9c8b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633889
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-22 11:05:57 +00:00
Dennis Kempin
191b95b5d7 native and aarch64 cross-compile containers
This CL adds the foundation for running tests consistently
in Kokoro and locally, for both x86 and aarch64.

The crosvm_builder is similar to the original image from
docker/crosvm.Dockerfile.
The main difference is that ChromeOS dependencies are not
compiled into the container, but built at runtime.

The crosvm_aarch64_builder installs the build enviornment
to cross-compile crosvm for aarch64. The tests are run
with user-space emulation using qemu-aarch64-static.

See ci/README.md for instructions on how to use these
builders.

Tests on aarch64 cannot all be run using user-space
emulation. We will need a VM to pass all smoke tests,
this work is tracked in b/177228167.

BUG=b:177133814
TEST=Tested by running
./ci/builder bin/smoke_test
./ci/builder cargo test
./ci/aarch64_builder cargo build
./ci/aarch64_builder cargo test -p tempfile

Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-20 17:41:27 +00:00