cargo fmt will only format those files that are referenced by
crosvm. To really catch all files in our source tree, we need to
run rustfmt directly.
This will check files multiple times, since rustfmt will follow
included modules. But is still faster that cargo fmt.
To run these in parallel, this change adds some tools to common.py
for parallel execution of commands.
This reduces the fmt check time from ~9s to 1.5s.
I verified that we are actually catching all files now by adding a
newline to all .rs files in our sources tree. The formatter reverted
them all.
BUG=b:218559722
TEST=./tools/fmt
Change-Id: If721d837ebc9eee3ce28fa7f439ab0bcc0b993cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3474926
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Add a framework for bindgen scripts.
This will be filled in when each binding is added in upcoming separate
commits.
BUG=b:218388029
TEST=None
Change-Id: I62363b49d92ceb5b54700589c2329daace7181b8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3339804
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
With crosvm adding support for windows, we want to make sure that
the line endings are not crlf.
git hooks and configs will be added as a part of another change.
Bug: b:213146393
Test: Modified Cargo.toml to have crlf endings and ran the script.
Change-Id: Ie6f53a7c0687578546e63770bb98706e4315a520
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3495708
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Adding some conditional compilation flags so we do not need to
exclude them on each test run. None of there actually increase
the number of tests run, but gets us closer to being able
to run `cargo test --workspace`
BUG=b:218374759
TEST=./tools/presubmit --all
Change-Id: I181a84847fc09a4e1a6fb2b5f21538ebc400d607
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3453046
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
As we make progress on upstreaming sys_util, we do not want more
dependencies on sys_util added. This check helps in that cause.
With this change only files that are inside common can depend directly
on sys_util.
Test: python3 ./tools/impl/check_code_hygiene.py common/sys_util_core
Bug: b:213149154
Change-Id: I57a8cb9189d3263a4031338b302cb27da799f4de
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3473344
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Run tests for sys_util_core, poll_token_derive and balloon_control on
windows.
Using dotfiles to disable/serialize test runs of a subset of crates does
not work well with third party crates as it forces us to commit the dot
file to the crate.
The patch modifies and uses the script that runs linux tests.
This patch also allows us to
- build/test child crate even if parent crate has disabled build/test.
- avoid building crosvm if it is not explicitly specified.
RIP short lived .windows_build_test_skip. You allowed us to run noop
kokoro tests.
Test: py .\tools\impl\test_runner.py --arch x86_64
Bug: b:215610772
Change-Id: Icc6d04ffd7c0c33d4f60aeac16fc7d23881c387d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3459809
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Use the new common.py utilities to make turn this bash script into
python. The behavior should be the same.
BUG=b:218559770
TEST=./tools/presubmit
Change-Id: Id1412c2076089fd21f280959d6cde1f4cb64e163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3469049
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tests can now be marked as large to increase the timeout from it's
original 60s to 120s.
The test runner is also updated to detect if the test is running in
an emulated environment, and doubles the timeout for those.
BUG=b:218681921
TEST=./tools/presubmit --all
Change-Id: I95fc238dcd23a5035826f422752eaf0ca215706c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3469496
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
We are replacing most of our bash tools with python versions so we
can support cross-platform development.
To facilitate writing shell-like scripts in Python, this CL adds
a common.py file to share common utilities. It curretly contains
tools to simplify shell-like command execution and argument parsing.
BUG=b:218559770
TEST=./tools/clippy && ./tools/fmt && ./tools/impl/common.py
Change-Id: I7f8b3523394973ed5c741b926fdc41e52133189f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3461240
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The crate requires special compilation flags that conflict with the
workspace flags.
We may want to consider moving it to common/ so the test runner can
compile it in a separate workspace.
This brings us one step closer to be able to run
`cargo build --workspace` with no excludes.
BUG=b:206689789
TEST=./tools/presubmit
Change-Id: Ice747ced00f81db0a8e05ff2bc43d4d0e323e456
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3453050
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
While we can run armhf binaries on aarch64 or use user-space emulation
to run aarch64 tests on the x86 host, some tests will fail in that
scenario as they will interact with kernel APIs directly (e.g. kvm).
This will make it easier to distinguish 'do not run on armhf because
we haven't fixed the tests' from 'this test is not supported on
a foreign arch kernel and that is expected'.
It also enables a quicker method of running aarch64 tests with
user-space emulation.
BUG=b:218374759
TEST=./tools/dev_container ./tools/run_tests --target=host
--arch=aarch64
Change-Id: I81871e11d3b02ee1ed3bac88bada28fbba8a3f31
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3447292
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Enable VVU feature unconditionally.
Since kokoro will start checking VVU code from this CL, clippy errors
are fixed.
In addition, the time limit of kokoro unit testing needed to be
increased because some time-consuming unit tests will start running on
aarch64 QEMU, which is really slow.
BUG=none
TEST=kokoro
Change-Id: I67437c060ff5ba9f690dcfc6e5c3dd155b95a785
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3450014
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
This will still run the same tests as before, the options are just
more consistent now.
BUG=b:218374759
TEST=./tools/presubmit
Change-Id: I41a8730f6cfaf65a62aaa26c45e2c284f0356d33
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3447541
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The patch disallows any platform specific code in sys_util_core and
doesn't let sys_util compile on windows platform.
This ensure to some extent that we keep sys_util and it's dependency
sys_util_core independent of windows code.
check_code_hygiene is not foolproof.
Test: Ran the script against a modified file in sys_util_core containing
string "target_os = "
Tried to compile sys_util on windows.
Bug: b:215610772
Upstream-Crate: common/win_sys_util
Cq-Depend: chromium:3433709
Change-Id: Ideb45092a959dd347d966633c3bd4e82f842b1a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3438709
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
sys_util_core won't have OS specific features so that sys_util and
win_sys_util can depend on it.
Test: Presubmit
Bug: b:213149154
Upstream-Crate: common/win_sys_util
Cq-Depend: chromium:3433709
Change-Id: I863f7a6bc7549944b4d114cca6d7be04c3093fc3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3426380
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
proc-macro tests run on the host, adding logic into the test runner
to accomodate for that.
BUG=b:218374759
TEST=./tools/presubmit
Change-Id: I83c853b9e28dae725d299509f7e3a07f56bd6ad8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3445531
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Adds use_uring checks to a few more test cases that are using the
URingExecutor.
BUG=b:218374759
TEST=./tools/run_tests
Change-Id: Ie5822dea4f22c2e8cffab13549c07873e89129d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3443634
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The tool is added to ./tools/fmt which will be called during
./tools/presubmit as well as by Kokoro during testing.
This requires a new dev container version to bundle the mdformat
tool in the container.
Note: mdformat does not have any IDE integrations, but the prettier
plugin with prose-wrap enabled and a line length of 100 provides
almost identical results.
BUG=None
TEST=./tools/fmt --check
Change-Id: I0ae5659eff8555df7c85c70e62095a1e116b98da
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3416098
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Two new clippy fixes are affecting us, but will be easy to fix in
a follow-up.
Fixed the dev-container so we can create a new version with the new
toolchain.
BUG=b:210037151
TEST=Kokoro
Change-Id: I9ac4d84aff72b1ee5219d6dab0a88667ca6c5951
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3328954
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Instead of enumerating crosvm crates separately, we can now use
the workspace to build all tests for crosvm.
This enables the vmm_vhost tests.
BUG=b:206026060
TEST=./tools/presubmit
Change-Id: I2af84ad8111e874ddea5e438bda41599caf1fd30
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3287463
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Developers may need to manually clean up the common/enumn directory - it
will be left behind if there are build artifacts (Cargo.lock, target
directory, etc.):
rm -rf common/enumn
BUG=b:205344148
TEST=cargo build
TEST=tools/presubmit
TEST=emerge-hatch crosvm # with https://crrev.com/c/3265967
Change-Id: I1af3bdd22f40e87895a78a5cbc8033476058c927
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3278774
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Building crates in common/ will try to access the file, but won't
be allowed to read it with CROS_WORKON_SUBTREE set.
BUG=b:195126527
TEST=./tools/run_tests --target=vm:aarch64
Change-Id: Ic455ca6c3102dd363daf70468e5d89003d10bb4c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3254042
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Reviewed-by: Abhishek Bhardwaj <abhishekbh@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
This change contains the results of running
./tools/contib/cargo_refactor.py
This will break the next uprev, and needs to be synchronizized
with the corresponding ebuild changes in https://crrev.com/c/3248925
BUG=b:195126527
TEST=./tools/run_tests
Change-Id: Ied15a1841887bb8f59fba65b912b81acf69beb73
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3248129
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Each of these is a separate workspace and can be built in
parallel.
BUG=b:195126527
TEST=./tools/run_tests
Change-Id: I9ad61b5d5d83903fda36c948ba470bba8f608d46
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3248128
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
When looking up crate options from test_options.py we used their
path relative to the crosvm root (e.g. common/p9). But when looking
up options for binaries, the name is parsed from cargo output and
just the crate name (e.g. p9). Fix this by just using the crate
name.
BUG=b:195126527
TEST=./tools/run_tests
Change-Id: I09a3469daa071dc66295d777449dc101a6564162
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3248127
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Includes some fixes for new clippy checks. A rebuilt
dev_container that ships the new toolchain.
This allows us to get rid of the annoying cargo clean
before running clippy.
BUG=b:203142205
TEST=./tools/presubmit
Change-Id: I9d486fbcf7b2d468f6a1375ac7df95091a2c1465
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3232277
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The new runner makes use of the simplified crosvm build and greatly
improves execution speed by gathering test binaries from cargo output
and executes them directly in parallel.
This allows all of our tests to execute in ~5 seconds when run locally.
The new test runner also makes use of the new testvm tools to make it
easy to switch between testing on the host, in a VM or via SSH on a
remote device.
See ./tools/run_tests --help for usage instructions.
To allow more iterative testing with the same test targets, this CL
includes a set_test_target script to write env vars that instruct cargo
to build for the target arch and run on the test target.
Note: The test runner can build for armhf, but we need build file fixed
to allow armhf to build successfully.
BUG=b:199951064
TEST=Inside ./tools/dev_container:
./tools/run_tests --target=host
./tools/run_tests --target=vm:aarch64
./tools/run_tests --aarch=armhf --build-only
./tools/set_test_target vm:aarch64 && source .envrc
cargo test
Change-Id: I60291fa726fb2298f62d1f032e55ed6e7ab1c4ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3221779
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
The new container just provides a standard debian box with a
rust toolchain.
Installation of dependencies has been extracted into scripts
which can also be used to set up a machine for development.
This will replace the current ./ci/ containers.
BUG=b:199950887,b:199950423
TEST=./tools/dev_container ./tools/run_tests
Change-Id: I832bc5b129246923df937a34614b4d74955304dc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3221781
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
We need to download a base image anyway, so we might as well make
that a full prebuilt of our VM image. This makes the initial
startup of ./tools/aarch64vm significantly faster. It also makes
it easier to add the prebuilt into a Docker dev container later.
Also move the VM data into the cargo target dir so we do not
dirty the src dir.
A Makefile for building and uploading the prebuilt has been added.
This change also moved the testvm code from ./tools/testvm into
./tools/impl so the python code can be shared with future scripts
(relative imports from other folders are tricky in Python).
BUG=b:199951064
TEST=./tools/aarch64vm ssh
Change-Id: I3fb57bfb8f7330b765e9be5ded821615ecddd841
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3221704
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
