Mount the cros root instead so the git worktree or symlink can be
resolved.
BUG=b:234322274
TEST=./tools/dev_container
Change-Id: I52d0cbe5e9a38cb6febcf1f328970e13209bcf9f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3768048
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
These will be useful in other scripts as well.
BUG=None
TEST=./tools/chromeos/mergebot update-dry-runs
Change-Id: Id75ed458fdd212d4564a547274209b1f39796ceb
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3765344
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
This was hard-coded to argh, which is the only package that uses it for
now, but the function may be used elsewhere in the future.
Also don't print a newline so the user can answer the prompt on the same
line after the [y/N].
BUG=None
TEST=Run tools/cl without python3-argh installed
Change-Id: I587fd1ded81f21724169a2fe90445f6055d1f421
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3764375
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Ask the user if they want to 'pip install' the dependency
if it is missing.
Not the prettiest solution, but helps make tools like
./tools/cl usable with minimal dependencies available.
Eventually we may consider pyenv or a similar solution.
BUG=b:239072471
TEST=pip uninstall argh; ./tools/cl
Change-Id: I4872e86883a9a0903a408d2f6d4f35dd1841f2ef
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3764425
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
BUG=None
TEST=tools/presubmit --all # with a failed test
Change-Id: I67262353cbd732b1394cdbe60b38d04c3a6e817b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3763359
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
This shuts off some of the Windows serial code beacuse it's blocked on a
separate CL that upstreams bits of the serial device. (It doesn't work
in this repo yet anyway.)
BUG=b:237597358
TEST=builds
Change-Id: If6b223ff29d55f44bff7d006c1e0b5db3733a4ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3741429
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
Some tests can fail if there is interference from other tests (e.g.
EINTR errors).
BUG=b:238232551
TEST=./tools/presubmit
Change-Id: I962a030771cee9d77f309d83fb7f0f43c18ada45
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3760897
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This is a reland of commit 213f9fe8a7.
In light of the upcoming VAAPI video decoder backend, add a VAAPI
wrapper crate that exposes a safe Rust API for a subset of the VAAPI C
code. This crate will be called from the VAAPI video decoder backend in
order to decode frames.
BUG=b:214478588
TEST=cargo build --features "video-decoder,vaapi"
TEST=`cargo test -- --include-ignored` in `media/libva` passes on a
device with Intel GPU and libva installed.
Change-Id: I586a160e477e466985c5cfa65a527542ddc03226
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3752274
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This reverts commit 213f9fe8a7.
Reason for revert: breaks the ChromeOS builder.
Original change's description:
> Add a VAAPI wrapper crate
>
> In light of the upcoming VAAPI video decoder backend, add a VAAPI
> wrapper crate that exposes a safe Rust API for a subset of the VAAPI C
> code. This crate will be called from the VAAPI video decoder backend in
> order to decode frames.
>
> BUG=b:214478588
> TEST=cargo build --features "video-decoder,vaapi"
> TEST=`cargo test -- --include-ignored` in `media/libva` passes on a
> device with Intel GPU and libva installed.
>
> Change-Id: I4afa96c49d045251827b97bd78faeba57575aedc
Bug: b:214478588
Change-Id: Ib5a88cd4c5fdd2df2e69fd3a0896ee789585840d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3752267
Auto-Submit: Alexandre Courbot <acourbot@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
In light of the upcoming VAAPI video decoder backend, add a VAAPI
wrapper crate that exposes a safe Rust API for a subset of the VAAPI C
code. This crate will be called from the VAAPI video decoder backend in
order to decode frames.
BUG=b:214478588
TEST=cargo build --features "video-decoder,vaapi"
TEST=`cargo test -- --include-ignored` in `media/libva` passes on a
device with Intel GPU and libva installed.
Change-Id: I4afa96c49d045251827b97bd78faeba57575aedc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3422779
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Normally we run test binaries in parallel, this change allows some to be
executed exclusively.
This fixes some of the odd issues and flakes we have been seeing.
integration_tests can hang if run in parallel with other tests and
cros_async has the odd habit of throwing the test_runner process into
the background.
Neither happens when the tests are the only ones being executed.
Also adds the capability to specify which test binaries to run,
which was helpful in debugging this issue.
BUG=b:238232551
TEST=./tools/run_tests --repeat 10
Change-Id: Ie5b82e329007399e95bf38111ec6e7cb6761ec0c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3751027
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
The test runner can now repeat tests in two ways:
- Via --repeat to run tests multiple times and fail if any one of them
fails. Which will help us find flakes in post-submit.
- Via --retry, which will retry a test if it has failed. Which makes
pre-submit tests more resilient to flakes.
Both can be configured by builder configs to adjust as needed.
Also slighly changes the repeat behavior to repeat tests in batches, so
we do not run the same test multiple times simultaneously, which can
cause some of them to fail. It's also easier to read the results.
BUG=b:238232551
TEST=added a random flake into some tests and used --repeat and --retry
Change-Id: I78e0ff0751da40a99a56080d9baf207307c9e93a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3751835
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This Allows Luci builders to cache cargo home between builds. So we
do not have to download so many third party crates with each build.
CARGO_HOME is specifically intended to be cached in CI systems.
BUG=b:233230027
TEST=CROSVM_CONTAINER_CACHE=/tmp/test ./tools/dev_container --clean
cargo build
Change-Id: I11580c5ed3151519ece4a651cb22d059c7c3eb87
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739368
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The format string wasn't actually formatted, so it just printed out the
literal "{shorthand}" string in the error message.
BUG=None
TEST=tools/run_tests --target=host --build-target=bogus
Change-Id: Idad47064db3112a37df3a630b92a4a4da8da82de
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3750069
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Updates run_tests to use cargo style target triples for specifying
build targets. A simple 'aarch64' or 'armhf' was nice while we just
had linux builds. We now are looking at windows and possibly
different toolchain options (e.g. msvc vs gnu), so our old system
was getting confusing and inconsistent.
We used to have some special handling for adding wrappers to test
runs for emulation (e.g. wine, qemu). That logic has been moved
into TestTarget which now contains not just where to run the test
but also how.
Supported are armhf/aarch64 qemu as well as wine64.
The CLI has been updated to match and now uses the build-target
argument instead of arch.
The following combinations have been tested (though not all
combinations actually pass all tests, which is a separate issue).
./tools/run_tests
./tools/run_tests --target=host --build-target=x86_64-unknown-linux-gnu
./tools/run_tests --target=host --build-target=armhf
./tools/run_tests --target=host --build-target=aarch64
./tools/run_tests --target=host --build-target=mingw64
./tools/run_tests --target=vm:aarch64
./tools/run_tests --target=vm:aarch64 --build-target=armhf
BUG=b:233914170
TEST=See above
Change-Id: Ic6dbb5b39788e2573714606d3bb0e7c712032d91
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739240
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
One more script that needs to run on python 3.8 so we can run test
for windows on luci.
BUG=b:234173142
TEST=vpython3 tools/run_tests
Change-Id: I0348a567b2edec3b5e3fca77f316f39d1e924adc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3723037
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
These libraries are needed to build the upcoming vaapi backend. Also
update the container so they are included.
BUG=b:214478588
TEST=./tools/dev_container ./tools/presubmit --all
Change-Id: Id30c38557d56b4c94ca5f2419c73b34520916ed0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3715087
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
To enable the merge bot on luci we had to solve a couple of problems:
- We cannot use http cookies for auth, so added gcloud auth into
merge_bot.
- Switch back to original HEAD after running merge bot. Otherwise
the version of the merge_bot script can change between invocations.
- Do not soft reset when checking out crosvm source.
- For less invasive testing, also added a few more exceptions when
using MERGE_BOT_TEST so it won't email, etc.
- Rename to update_chromeos_merges, as it is more fitting to what
the bot is doing
BUG=b:233913643
TEST=https://ci.chromium.org/swarming/task/5b58ed4497fda510
Change-Id: Iaa9b4821b70a1e90d19637d01856196bd7852ed5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3692686
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
The virtio video decoder device is currently only available under very
drastic conditions: a build linked against libvda (a ChromeOS-only
library that needs the cros chroot to be built and linked against), and
a ChromeOS-flavored Chrome instance running alongside crosvm, so the
browser can provide the video decoding service through Mojo.
This makes the decoder device very difficult to develop on for
non-Chromies, and also for Chromies actually since they will always need
a DUT to test it on.
This patch introduces an alternative decoder backend based on
ffmpeg's libraries that performs decoding on the host's CPU. It supports both
guest pages and virtio objects as target, and can be considered a
reliable and predictable way to test the decoder in any environment.
We introduce our own ffmpeg bindings after a quick state of the art
revealed that the existing ones were all unsuitable, either for
technical or licensing reasons. Doing so is also not a big effort and
does not add any new external crate dependency to crosvm.
BUG=b:169295147
TEST=cargo test --features "video-decoder,ffmpeg" -p devices ffmpeg
TEST=v4l2r's simple_decoder example decodes test-25fps.h264 properly with the
following command:
./simple_decoder test-25fps.h264 /dev/video0 --input_format h264 --save test-25fps.nv12
TEST=ARCVM Android youtube plays videos correctly when the ffmpeg
backend is used.
Change-Id: Ic9c586193f7939f2a3fe59d009c3666585a8bbc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3026355
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Add the ffmpeg libraries needed by the ffmpeg decoder device backend and
bump the dev_container and testvm to include them.
BUG=b:169295147
TEST=./tools/install-deps
TEST=./tools/dev_container
Change-Id: Ifc07b9599403aa1ed18a96067ada1fa2efa6e13c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3685682
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
See USAGE for documentation.
This tool will be needed to upload to the upstream repository, as
we will no longer be able to use repo.
Currently, for testing it still points to the chromiumos repository.
BUG=b:221088786
TEST=Follow USAGE
Change-Id: I4a8d88a8354942f0ddfb7f9420ef1cf7f5885867
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3687419
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
They have been dropped with the recent refactoring to not use argh.
BUG=None
TEST=./tools/dev_container -v
Change-Id: Ifa65b14d316e27dbb5cef9714fbd3adf110b79e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3664051
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This fixes little annoyances where fmt or other tools would try to run
on files not part of version control (e.g. local editor configs).
BUG=b:227224259
TEST=./tools/fmt -v
Change-Id: Iddff90ec95a9e28cd80ac3aece26202e8a42e9d3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3662022
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This script is naturally used outside of the container and we
want to avoid having to install dependencies or special python
versions to run it.
This CL reduces the python requirement from 3.9 to 3.8 and
allows the script to run without any dependencies.
BUG=b:233230344
TEST=pyenv global 3.8.15
./tools/dev_container
Change-Id: Ieeb868f7fb275a8e6e39e577b5e984895307765e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657816
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
This reverts commit 15a4ee3200.
Reason for revert: Should pass now
Original change's description:
> Do not run cros_async tests on armhf
>
> The test has been flaking with segfaults occasionally.
>
> BUG=None
> TEST=presubmit
>
> Change-Id: Ifbfee7e7ed5de91b7bdf9b25db7983139951f81f
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606751
> Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
> Tested-by: kokoro <noreply+kokoro@google.com>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>
Bug: None
Change-Id: I85d0885ce9f9fa3d46dd86536a5a20a71ffbbabb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3626010
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
The sys_util crate has been merged into base, but a few places still
refer to the old name. Fix or remove them as appropriate.
BUG=b:227226222
TEST=tools/presubmit --all
Change-Id: Icf9b57aff672b7c1afec768c9694e059f0f9a43d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3621205
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Fixes cases when we use target vm:aarch64 but arch armhf and test_runner
picks up tests for aarch64
BUG=none
TEST=run and made sure armhf-disabled tests are not picked up
Change-Id: I4d0734bb6320fc698bf16adc59de5e03b00a4a68
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3617081
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Anton Romanov <romanton@google.com>
The test has been flaking with segfaults occasionally.
BUG=None
TEST=presubmit
Change-Id: Ifbfee7e7ed5de91b7bdf9b25db7983139951f81f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606751
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
And fix a nit in the existing tests to make it pass.
BUG=b:219965702
TEST=presubmit
Change-Id: I65fb98b04d7042755930dd34913f679fa129c88c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3594248
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This change adds python type and formatting checks and
consolidates code health checks in ./tools/health-check.
Dealing with relative imports in python is tricky, so
we are making ./tools/impl a proper package with no
directly executable files.
Some of the bash shorthands in ./tools had to be converted
to python for this.
To make the new checks pass, we run the formatter and fix
some mypy type checks.
TEST=./tools/health-check
BUG=b:218559722,b:219965702
Change-Id: Ie18d3d6dd2f5a033141e167a6e1aa762791941d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3558592
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This would previously trigger a program not found error.
BUG=None
TEST=Added doctest
Change-Id: I357c44d2b9ce7b2a3240958c5c88008a29164baa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3587885
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
If the value is not set, treat it as empty.
BUG=None
TEST=Tested by removing the config setting locally.
Change-Id: Ia160285c03bffcf7a332f3891587e7ada2228623
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585860
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
The new merge bot is written in python to allow us to improve the
merge process.
The new process is as follows:
- The merge bot will only create a new merge when the previous
ones have been submitted.
- When the bot creates a new merge, it'll split the merge commits
into smaller changes to make bisection easier.
This allows us to only ever deal with one set of merge commits
instead of piling on new merges day after day.
For dry runs, the new process is:
- The bot will check existing dry runs and abandon them if the CQ
is done.
- The bot will vote Verified+-1 to indicate if the dry run passed.
- If no more dry runs are active, it'll create a new one.
This ensures we always have a dry run in the CQ and can easily see
in gerrit which ones have passed.
The python script will also allow us to add convenience features
for oncall for retrying bratches, etc.
BUG=b:227347397
TEST=The script can be run locally:
./tools/chromeos/merge_bot update-merges
./tools/chromeos/merge_bot update-dry-runs
Change-Id: I51fc0741f5032868ad07f5e926e570e29cc461fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583254
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
