Updates run_tests to use cargo style target triples for specifying
build targets. A simple 'aarch64' or 'armhf' was nice while we just
had linux builds. We now are looking at windows and possibly
different toolchain options (e.g. msvc vs gnu), so our old system
was getting confusing and inconsistent.
We used to have some special handling for adding wrappers to test
runs for emulation (e.g. wine, qemu). That logic has been moved
into TestTarget which now contains not just where to run the test
but also how.
Supported are armhf/aarch64 qemu as well as wine64.
The CLI has been updated to match and now uses the build-target
argument instead of arch.
The following combinations have been tested (though not all
combinations actually pass all tests, which is a separate issue).
./tools/run_tests
./tools/run_tests --target=host --build-target=x86_64-unknown-linux-gnu
./tools/run_tests --target=host --build-target=armhf
./tools/run_tests --target=host --build-target=aarch64
./tools/run_tests --target=host --build-target=mingw64
./tools/run_tests --target=vm:aarch64
./tools/run_tests --target=vm:aarch64 --build-target=armhf
BUG=b:233914170
TEST=See above
Change-Id: Ic6dbb5b39788e2573714606d3bb0e7c712032d91
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739240
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
One more script that needs to run on python 3.8 so we can run test
for windows on luci.
BUG=b:234173142
TEST=vpython3 tools/run_tests
Change-Id: I0348a567b2edec3b5e3fca77f316f39d1e924adc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3723037
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
These libraries are needed to build the upcoming vaapi backend. Also
update the container so they are included.
BUG=b:214478588
TEST=./tools/dev_container ./tools/presubmit --all
Change-Id: Id30c38557d56b4c94ca5f2419c73b34520916ed0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3715087
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
To enable the merge bot on luci we had to solve a couple of problems:
- We cannot use http cookies for auth, so added gcloud auth into
merge_bot.
- Switch back to original HEAD after running merge bot. Otherwise
the version of the merge_bot script can change between invocations.
- Do not soft reset when checking out crosvm source.
- For less invasive testing, also added a few more exceptions when
using MERGE_BOT_TEST so it won't email, etc.
- Rename to update_chromeos_merges, as it is more fitting to what
the bot is doing
BUG=b:233913643
TEST=https://ci.chromium.org/swarming/task/5b58ed4497fda510
Change-Id: Iaa9b4821b70a1e90d19637d01856196bd7852ed5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3692686
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
The virtio video decoder device is currently only available under very
drastic conditions: a build linked against libvda (a ChromeOS-only
library that needs the cros chroot to be built and linked against), and
a ChromeOS-flavored Chrome instance running alongside crosvm, so the
browser can provide the video decoding service through Mojo.
This makes the decoder device very difficult to develop on for
non-Chromies, and also for Chromies actually since they will always need
a DUT to test it on.
This patch introduces an alternative decoder backend based on
ffmpeg's libraries that performs decoding on the host's CPU. It supports both
guest pages and virtio objects as target, and can be considered a
reliable and predictable way to test the decoder in any environment.
We introduce our own ffmpeg bindings after a quick state of the art
revealed that the existing ones were all unsuitable, either for
technical or licensing reasons. Doing so is also not a big effort and
does not add any new external crate dependency to crosvm.
BUG=b:169295147
TEST=cargo test --features "video-decoder,ffmpeg" -p devices ffmpeg
TEST=v4l2r's simple_decoder example decodes test-25fps.h264 properly with the
following command:
./simple_decoder test-25fps.h264 /dev/video0 --input_format h264 --save test-25fps.nv12
TEST=ARCVM Android youtube plays videos correctly when the ffmpeg
backend is used.
Change-Id: Ic9c586193f7939f2a3fe59d009c3666585a8bbc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3026355
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Add the ffmpeg libraries needed by the ffmpeg decoder device backend and
bump the dev_container and testvm to include them.
BUG=b:169295147
TEST=./tools/install-deps
TEST=./tools/dev_container
Change-Id: Ifc07b9599403aa1ed18a96067ada1fa2efa6e13c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3685682
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
See USAGE for documentation.
This tool will be needed to upload to the upstream repository, as
we will no longer be able to use repo.
Currently, for testing it still points to the chromiumos repository.
BUG=b:221088786
TEST=Follow USAGE
Change-Id: I4a8d88a8354942f0ddfb7f9420ef1cf7f5885867
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3687419
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
They have been dropped with the recent refactoring to not use argh.
BUG=None
TEST=./tools/dev_container -v
Change-Id: Ifa65b14d316e27dbb5cef9714fbd3adf110b79e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3664051
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This fixes little annoyances where fmt or other tools would try to run
on files not part of version control (e.g. local editor configs).
BUG=b:227224259
TEST=./tools/fmt -v
Change-Id: Iddff90ec95a9e28cd80ac3aece26202e8a42e9d3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3662022
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This script is naturally used outside of the container and we
want to avoid having to install dependencies or special python
versions to run it.
This CL reduces the python requirement from 3.9 to 3.8 and
allows the script to run without any dependencies.
BUG=b:233230344
TEST=pyenv global 3.8.15
./tools/dev_container
Change-Id: Ieeb868f7fb275a8e6e39e577b5e984895307765e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657816
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
This reverts commit 15a4ee3200.
Reason for revert: Should pass now
Original change's description:
> Do not run cros_async tests on armhf
>
> The test has been flaking with segfaults occasionally.
>
> BUG=None
> TEST=presubmit
>
> Change-Id: Ifbfee7e7ed5de91b7bdf9b25db7983139951f81f
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606751
> Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
> Tested-by: kokoro <noreply+kokoro@google.com>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>
Bug: None
Change-Id: I85d0885ce9f9fa3d46dd86536a5a20a71ffbbabb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3626010
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
The sys_util crate has been merged into base, but a few places still
refer to the old name. Fix or remove them as appropriate.
BUG=b:227226222
TEST=tools/presubmit --all
Change-Id: Icf9b57aff672b7c1afec768c9694e059f0f9a43d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3621205
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Fixes cases when we use target vm:aarch64 but arch armhf and test_runner
picks up tests for aarch64
BUG=none
TEST=run and made sure armhf-disabled tests are not picked up
Change-Id: I4d0734bb6320fc698bf16adc59de5e03b00a4a68
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3617081
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Anton Romanov <romanton@google.com>
The test has been flaking with segfaults occasionally.
BUG=None
TEST=presubmit
Change-Id: Ifbfee7e7ed5de91b7bdf9b25db7983139951f81f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606751
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
And fix a nit in the existing tests to make it pass.
BUG=b:219965702
TEST=presubmit
Change-Id: I65fb98b04d7042755930dd34913f679fa129c88c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3594248
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This change adds python type and formatting checks and
consolidates code health checks in ./tools/health-check.
Dealing with relative imports in python is tricky, so
we are making ./tools/impl a proper package with no
directly executable files.
Some of the bash shorthands in ./tools had to be converted
to python for this.
To make the new checks pass, we run the formatter and fix
some mypy type checks.
TEST=./tools/health-check
BUG=b:218559722,b:219965702
Change-Id: Ie18d3d6dd2f5a033141e167a6e1aa762791941d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3558592
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This would previously trigger a program not found error.
BUG=None
TEST=Added doctest
Change-Id: I357c44d2b9ce7b2a3240958c5c88008a29164baa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3587885
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
If the value is not set, treat it as empty.
BUG=None
TEST=Tested by removing the config setting locally.
Change-Id: Ia160285c03bffcf7a332f3891587e7ada2228623
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585860
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
The new merge bot is written in python to allow us to improve the
merge process.
The new process is as follows:
- The merge bot will only create a new merge when the previous
ones have been submitted.
- When the bot creates a new merge, it'll split the merge commits
into smaller changes to make bisection easier.
This allows us to only ever deal with one set of merge commits
instead of piling on new merges day after day.
For dry runs, the new process is:
- The bot will check existing dry runs and abandon them if the CQ
is done.
- The bot will vote Verified+-1 to indicate if the dry run passed.
- If no more dry runs are active, it'll create a new one.
This ensures we always have a dry run in the CQ and can easily see
in gerrit which ones have passed.
The python script will also allow us to add convenience features
for oncall for retrying bratches, etc.
BUG=b:227347397
TEST=The script can be run locally:
./tools/chromeos/merge_bot update-merges
./tools/chromeos/merge_bot update-dry-runs
Change-Id: I51fc0741f5032868ad07f5e926e570e29cc461fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583254
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Also fixes few warning introduced during upstream which linux clippy
run does not catch.
test: py tools\clippy
bug: 226966790
Change-Id: I6979f87be5d6d34727afabafcaccb57631975567
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3553401
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Does couple of things
- move tube under right directory
- cleanup cros_async dependency
- export tube module from base
- enable base windows tests
- add verbose flag to test runner
test: built on window and presubmit
bug: b:213151429
bug: b:213153157
Change-Id: I0d1906d7edcc3352db53325ea7527550eaf00895
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3550296
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
install-deps is now a Python script, update the message to make user's
life easier.
BUG=None
TEST=./tools/presubmit
Change-Id: Id1eca4f5b1fb0066574b6b7c381c908e65d919e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3536892
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dmitry Torokhov <dtor@chromium.org>
This will allow us to merge sys_util and win_sys_util.
The original source locations will remain until they
are moved into the ChromeOS source base.
See go/future-of-sys-util for details
BUG=b:22320646
TEST=run it
Change-Id: Ie1adcadba9d35f18cc7472dc78f05262114a726f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3536890
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This is a prerequisite for sys_util* to move into the base
crate. The crate should never have been in common in the
first place since it is not shared.
BUG=b:223206469
TEST=python tools/contrib/cargo_refactor.py
Change-Id: I60115e0418c6980d7bf7ff624a3cc0a24e71a57d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3530502
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
cargo fmt will only format those files that are referenced by
crosvm. To really catch all files in our source tree, we need to
run rustfmt directly.
This will check files multiple times, since rustfmt will follow
included modules. But is still faster that cargo fmt.
To run these in parallel, this change adds some tools to common.py
for parallel execution of commands.
This reduces the fmt check time from ~9s to 1.5s.
I verified that we are actually catching all files now by adding a
newline to all .rs files in our sources tree. The formatter reverted
them all.
BUG=b:218559722
TEST=./tools/fmt
Change-Id: If721d837ebc9eee3ce28fa7f439ab0bcc0b993cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3474926
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Add a framework for bindgen scripts.
This will be filled in when each binding is added in upcoming separate
commits.
BUG=b:218388029
TEST=None
Change-Id: I62363b49d92ceb5b54700589c2329daace7181b8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3339804
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
With crosvm adding support for windows, we want to make sure that
the line endings are not crlf.
git hooks and configs will be added as a part of another change.
Bug: b:213146393
Test: Modified Cargo.toml to have crlf endings and ran the script.
Change-Id: Ie6f53a7c0687578546e63770bb98706e4315a520
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3495708
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Adding some conditional compilation flags so we do not need to
exclude them on each test run. None of there actually increase
the number of tests run, but gets us closer to being able
to run `cargo test --workspace`
BUG=b:218374759
TEST=./tools/presubmit --all
Change-Id: I181a84847fc09a4e1a6fb2b5f21538ebc400d607
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3453046
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
As we make progress on upstreaming sys_util, we do not want more
dependencies on sys_util added. This check helps in that cause.
With this change only files that are inside common can depend directly
on sys_util.
Test: python3 ./tools/impl/check_code_hygiene.py common/sys_util_core
Bug: b:213149154
Change-Id: I57a8cb9189d3263a4031338b302cb27da799f4de
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3473344
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
