Commit graph

157 commits

Author SHA1 Message Date
Vikram Auradkar
1b32084a56 crosvm: Enable default features on windows.
The patch makes usb feature no-op on windows as USB pass-through is
not supported on windows.

BUG=b:241251677
BUG=b:213149155
TEST=tools/presubmit

Change-Id: Id82d4c732a86e782695d2af698cc08e8e3fd2d35
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4006819
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2022-11-07 23:03:25 +00:00
Ryuichiro Chiba
df7a2e51bf e2e_tests: Add a tool to dev_container and guest rootfs
Add a tool used in vsock tests to dev_container and guest rootfs

BUG=b:254375643
TEST=`./tools/presubmit` with the new guest rootfs on the new
dev_container

Change-Id: I135e6d2d413bedffef0fd69edd4ab14080429e3a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3974349
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Takaya Saeki <takayas@chromium.org>
Commit-Queue: Ryuichiro Chiba <chibar@chromium.org>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
2022-11-07 04:00:40 +00:00
Dennis Kempin
7eadfac122 vhost: Move tests into an integration test
The tests require access to vhost syscalls which are not available
in all environments and are not suitable for unit tests.

BUG=b:244623767
TEST=presubmit

Change-Id: I2b6320d005da8966adab3f016237fedf1361055a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4004981
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2022-11-04 06:49:36 +00:00
Dennis Kempin
3375878f14 Move hypervisor::kvm tests to integration tests
These tests require kvm to pass and are not suitable for a unit
test.

In order to move the test, KvmVm::create_vcpu had to be renamed
to prevent conflicts with the trait function VcpuX86_64::create_vcpu,
implemented for the same type.

The aarch64 variant does not actually run in CI since the tests
do not pass in the emulated aarch64 environment. So the
DO_NOT_RUN_AARCH64 flag remains.

BUG=b:244623454
TEST=presubmit

Change-Id: I79bba3926a38d19350e2fd3c7bfa4662499223e5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3999799
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-11-03 23:19:11 +00:00
Dennis Kempin
b50f6580a2 test_config: Cleanup and enable some video crates
Some of these flags were no longer necessary since we now split
between unit and integration tests.
Others could be enabled by adding some missing dependencies.

BUG=None
TEST=presubmit

Change-Id: Ife5b09a4ee8b27885ee2042d5d93447e8d4deedd
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4004345
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-11-03 23:03:54 +00:00
Dennis Kempin
009cb53c76 Rename integration_tests to e2e_tests
Cargo already uses the term 'integration test' for tests living
in the tests/ directory of a crate.

To prevent confusion, rename our 'integration_tests' crate to
'e2e_tests'.

BUG=None
TEST=presubmit

Change-Id: Icfa819eaed08be54ab0f36092f1ba367f3f1ce88
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4004977
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
2022-11-03 22:49:39 +00:00
Dennis Kempin
389a02462d dev_container: Install tmux and mold
tmux can be used to run presubmit checks in parallel:

  $ tools/presubmit --tmux

And mold is a faster linker that will be used for experiments.

BUG=None
TEST=None

Change-Id: Ie332d7285aa94d9072e201a3cac825337ae825cb
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3997872
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-11-02 01:56:48 +00:00
Dennis Kempin
b28d5a963d Fix winetricks install and build new container
Uprevs the container version to include the newly added
winetricks.

Fixes /dev/kvm permissions.

BUG=None
TEST=CQ

Change-Id: I019178f103068d3ae0dc048180a14c8786e738e0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3994884
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2022-11-01 19:04:35 +00:00
Dennis Kempin
45b5c13047 Make common crates part of the crosvm workspace
This greatly simplifies and speeds up compilation
and clippy times.

This ChromeOS-side building these crates has been
updated to copy the source out of the source tree
so that they will compile as a separate workspace.

BUG=b:256020427
TEST=presubmit

Change-Id: I2e0f1f6724924d6bdd70ea28d7777df7966cf724
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3988324
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-10-31 21:33:33 +00:00
Dennis Kempin
278e5ffeba test_runner: Add separate cargo target dir for each platform
Using the same CARGO_TARGET_DIR for all builds prevents us from
building for platforms in parallel, since cargo will lock the
directory to only run one build at a time.

Use the same directory for clippy as well, and ensure that
clippy won't invalidate caches created by run_tests.

This reduces the build time for tools/presubmit by about 50%.

Another advantage is that rust_analyzer and run_tests will no longer
block each other or invalidate the cache when run with different
feature flags.

Note: This introduces two subtle changes to the build that required nit
fixes:
- build.rs files are now run through clippy as well
- common/* crates are now also built for the target architecture instead
  the host.

BUG=None
TEST=tools/presubmit

Change-Id: I8da9ef53418c0b15827d512a04e77828621aef88
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3984416
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-10-31 21:33:33 +00:00
Dennis Kempin
06c98828c0 run_tests: Allow integration tests to run natively on windows
This should re-enable windows integration tests in CI.

BUG=b:255366812
TEST=CQ

Change-Id: I869e6b17ef00bdcbbd32758787345ea893996a7f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3988894
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2022-10-29 14:53:14 +00:00
Vikram Auradkar
2653304c8e tools: Install winetricks
This will help us running audio tests under wine.

BUG=b:237011316
TEST=presubmit

Change-Id: Iba297159291abd135fb1972a19fa5b5c216fa956
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971028
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-10-27 18:11:46 +00:00
Zihan Chen
988858b66b infra: Add build_chromeos_container builder
Add a new builder to build crosvm in crOS tree, and all the
depencies of this new builder.

BUG=b:240692674
TESTED=led get-builder luci.crosvm.ci:chromeos_amd64-generic | led edit-cr-cl https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3966928 | led edit-recipe-bundle | led edit -r build_chromeos_hatch | led launch

Change-Id: Id2f284139922916edd2dd584f576da9fb3445518
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3966928
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2022-10-26 23:05:08 +00:00
Dennis Kempin
bf4cad131d reland: dev_container: Fix podman and enable unprivileged containers
reland note: Added wineboot for Dockerfile.user, since it prepares
directories for wine on a per-user basis.

To enable podman, the Dockerfile has been split into a root
run Dockerfile and one that adds a non-root user.

The following combinations have been tested:

./tools/dev_container -v --clean --podman --unprivileged
./tools/dev_container -v --clean --podman
./tools/dev_container -v --clean --unprivileged

And warnings have been added to ensure users are aware that
the only fully supported variant is running a privileged
docker container:

./tools/dev_container -v --clean

The unprivileged containers will allow us to validate if
unit tests require privileged system access.

BUG=None
TEST=See above

Change-Id: Ifd70c1e30ef266e39bf517e315dc88fccecc8a62
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3983255
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Zihan Chen <zihanchen@google.com>
2022-10-26 20:10:25 +00:00
Dennis Kempin
2f5eb3ac64 Extract devices integration tests
This change moves most ircchip tests into an integration test.
These tests rely on kvm, and if they do not - reuse much of the
test code from each other, so they need to move together.

Note: This removes the apic_timer test. The test has been disabled
for a while due to it's use of sleep in unit tests.

We cannot support it as an integration test either, since it combines
the sleep with a FakeClock. userspace.rs swaps the real clock for
FakeClock when compiled with cfg(test), but integration tests do not
compile with cfg(test), so we cannot use the FakeClock.

The biggest side-effect is faster execution as we can run all other 300+
tests in parallel and via user-space emulation, significantly cutting
down on the test times. It also allows those tests to run in a
podman container.

BUG=b:244620308
TEST=CQ

Change-Id: I1728a736d27e924daf228752711435885dacfa6a
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3977111
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-10-26 17:53:08 +00:00
Dennis Kempin
a0e6a664fc Revert "dev_container: Fix podman and enable unprivileged containers"
This reverts commit 6a2b1fda85.

Reason for revert: Broke mingw64 builds

Original change's description:
> dev_container: Fix podman and enable unprivileged containers
>
> To enable podman, the Dockerfile has been split into a root
> run Dockerfile and one that adds a non-root user.
>
> The following combinations have been tested:
>
> ./tools/dev_container -v --clean --podman --unprivileged
> ./tools/dev_container -v --clean --podman
> ./tools/dev_container -v --clean --unprivileged
>
> And warnings have been added to ensure users are aware that
> the only fully supported variant is running a privileged
> docker container:
>
> ./tools/dev_container -v --clean
>
> The unprivileged containers will allow us to validate if
> unit tests require privileged system access.
>
> BUG=None
> TEST=See above
>
> Change-Id: I185b1d9c3829674986305b0e72a39b1a4ba11b98
> Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971029
> Reviewed-by: Zihan Chen <zihanchen@google.com>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>
> Reviewed-by: Dennis Kempin <denniskempin@google.com>

Bug: None
Change-Id: Id57686ed869abcfb54431aa328c54234b9465eb7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3979385
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
2022-10-26 03:06:26 +00:00
Dennis Kempin
6a2b1fda85 dev_container: Fix podman and enable unprivileged containers
To enable podman, the Dockerfile has been split into a root
run Dockerfile and one that adds a non-root user.

The following combinations have been tested:

./tools/dev_container -v --clean --podman --unprivileged
./tools/dev_container -v --clean --podman
./tools/dev_container -v --clean --unprivileged

And warnings have been added to ensure users are aware that
the only fully supported variant is running a privileged
docker container:

./tools/dev_container -v --clean

The unprivileged containers will allow us to validate if
unit tests require privileged system access.

BUG=None
TEST=See above

Change-Id: I185b1d9c3829674986305b0e72a39b1a4ba11b98
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971029
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-10-26 02:06:06 +00:00
Dennis Kempin
b896b869e4 base: Extract integration tests
The extracted tests rely on access to system devices
or global state that prevent them from being run in parallel
or in restricted environments.

As an integration test they will be executed separately and
single threaded.

Updates the test runner to ensure integration tests are actually
run single threaded as intended.

BUG=b:244623061
TEST=./tools/run_tests base:\* --repeat 100 -p x86_64/mingw64/aarch64

Change-Id: I4267b9f79055208aca86796d902da251816bcada
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3971025
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-10-24 22:49:29 +00:00
Zihan Chen
a95adc3124 tools/cros_container: Add experimental CrOS builder container
This allow GCP cloud build to fresh and incrementally build updated
docker containers to test upstream crosvm in CrOS tree efficiently.
Currently this resides in an experimental GCP project while being WIP.

TEST=Both fresh and incremental can build, emerge-hatch in pulled container
BUG=b:240692674

Change-Id: Id54014f347c00e4dae16674b430159dd8cfa67aa
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3934983
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-10-17 21:29:57 +00:00
Dennis Kempin
5a32a2d198 Enable vaapi feature upstream
The feature can be built but not tested without access to an intel
GPU.
To enable the build, the testvm and dev container are updated to
include libva.

BUG=b:244619376
TEST=presubmit

Change-Id: Ia8c4dc46ccbcd244bf57441f2c550a6b73c67b1c
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3946027
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
2022-10-14 20:31:19 +00:00
Dennis Kempin
bb9a3a5572 Document feature flags and introduce new feature sets
The feature flags are documented using the document_features
crate.
Each platform gets one feature set that enables the features that
are built and tested for that platform.

The only functional difference is that the plugin feature is now
enabled in clippy. Otherwise this should be a no-op.

BUG=b:243894033
TEST=presubmit && cargo doc

Change-Id: Ia910bc2670696172daedcc503f7ad5844a844964
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3946024
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-10-14 20:31:19 +00:00
Vikram Auradkar
44a07970cb sandbox: upstream sandbox
The crate depends on a prebuilt library.

BUG=b:213170185
TEST=presubmit

Change-Id: I810baa14afa53e88f6b2a739481a6d3656efc2e5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3784338
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-10-12 21:05:39 +00:00
Vikram Auradkar
1acc0a28d9 prebuilts: move prebuilts tests under tests
BUG=b:246987526
TEST=presubmit

Change-Id: I86eed7c84f7e21eb7155695489ee9446f999dc1d
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3936733
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2022-10-07 15:54:37 +00:00
Dennis Kempin
8a38e8f36e Re-enable video-decoder feature
This was disabled in https://crrev.com/c/3835500 due to the lack
of ffmgeg 5.0 in our dev containers.
The container has since been upgraded in https://crrev.com/c/3892621

BUG=b:244619658
TEST=presubmit

Change-Id: Ia2bd41fbd780bad7d7706e0ffe7554e202ec0eb1
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3907377
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-10-06 18:28:56 +00:00
Vikram Auradkar
eb52905d36 prebuilts: Add helper crate to download prebuilts
BUG=b:246987526
TEST=presubmit

Change-Id: I1e54cf4a176d48c30340d6e7925733d6a1f4ce93
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3784337
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
2022-10-04 20:22:47 +00:00
Zihan Chen
646b6ea8fa crosvm-fuzz: Migrate crosvm-fuzz to cargo fuzz
Fuzzer targets can be ran with
cargo +nightly fuzz run --fuzz-dir crosvm-fuzz --features
upstream-fuzz <target>

This should enable us to move fuzzing to anywhere including
ClusterFuzz while maintain compatibility with cros infra.

TEST=`cargo fuzz` won't crash in first 30s,
`USE="asan fuzzer" emerge-hatch crosvm` builds,
`/build/hatch/usr/libexec/fuzzers/crosvm_qcow_fuzzer` won't
crash in first 30s
FIXED=b:245007212
BUG=b:244631591

Change-Id: I4b262ee1a6a90247dea96347c55a3849af793bec
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3905095
Auto-Submit: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-10-03 16:12:15 +00:00
Dennis Kempin
97da569f79 tools/run_tests: Split unit tests from integration tests
Changes how unit tests are executed. Instead of running all tests on
a selected target (i.e. host or a VM), we will select a platform to test
and run_tests will use a separate target for unit tests than for
integration tests.

e.g. instead of running all tests in the aarch64 vm:

./tools/run_tests --target=vm:aarch64

We can now run

./tools/run_tests --platform=aarch64

to run unit tests via user-space emulation on the host, and only
integration tests on the VM.

This will eventually apply to x86 testing as well, so tests can be
run in an unprivileged environment (e.g. podman).

A new TestOption has been added to mark unit tests that have system
interactions that require them to be run like integration tests in a VM.
These should be fixed.

You can also use --unit-tests and --integration-tests to run just one
type of tests. Running unit tests only takes a few seconds on most
platforms since we can use user-space emulation.

BUG=b:247139912
TEST=./tools/run_tests
./tools/run_tests -p aarch64
./tools/run_tests -p mingw64
./tools/run_tests -p armhf

Change-Id: Icd0c502623f1889906d199e752b3eccb7de76dc0
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3902688
Reviewed-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-09-20 18:27:23 +00:00
Dennis Kempin
acd262cb42 tools/x86vm: Small fixes to VM
- Check if ssh port is available before starting VM
- Move data dir to /tmp. Otherwise we may delete running images
  or pid files with `cargo clean`.
- Change CPU type to prevent vmx warnings.

BUG=247139912
TEST=./tools/run_tests --target=vm:x86_64

Change-Id: Id948ab9d67b2dddce8da663bb00b8924bc26c041
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3902687
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zihan Chen <zihanchen@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-19 22:04:20 +00:00
Dennis Kempin
aae0141680 dev_container: Upgrade to debian bookworm
Debian bullseye is becoming old and we require a couple of newer
dependencies. gLinux is also tracking bookworm, so we are staying
close to our usual development environment.

Since the official rust images do not have a bookworm version,
we switch to the official debian image and add a rustup install
to ./tools/install-deps.

The new glibc version uses clone3 in multiple devices, adding
this new syscall to our policy to pass integration tests.

Drive-by change: Upgrading rust-toolchain from 1.62.0 to 1.62.1

BUG=b:243081643
TEST=CQ

Change-Id: I8af721ed4a83df61163d67001b777166abe8abfa
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3892621
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-14 22:32:36 +00:00
Daniel Verkamp
74f1ca4e78 bindgen: update bindings to Linux 5.15
This requires a few tweaks to non-generated code:
- VIRTIO_ID_VIDEO_ENC/DEC -> ENCODER/DECODER
- io_uring unnamed union layout change

Change-Id: I58e118efa5c6bf28ff56d211fec5603651cb60bc
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3893753
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2022-09-14 22:09:30 +00:00
Daniel Verkamp
7c6d8bec3f health-check: enforce blank line after copyright
While we are tweaking all of the copyright headers, let's take the
opportunity to ensure there is always a blank line after the copyright
header for consistency. (Almost all files already follow this style.)

This includes a slightly ugly regex to allow the end of a C-style
comment block after the end of the copyright:

/*
 * Example comment block
 */   <-- this line

Change-Id: Idfd0855861e5ecb3d33afae942fdba908af0dcff
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3892521
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2022-09-13 22:24:35 +00:00
Dennis Kempin
1dab58a2cf Update all copyright headers to match new style
This search/replace updates all copyright notices to drop the
"All rights reserved", Use "ChromiumOS" instead of "Chromium OS"
and drops the trailing dots.

This fulfills the request from legal and unifies our notices.

./tools/health-check has been updated to only accept this style.

BUG=b:246579983
TEST=./tools/health-check

Change-Id: I87a80701dc651f1baf4820e5cc42469d7c5f5bf7
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3894243
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-09-13 18:41:29 +00:00
Dennis Kempin
78f62a44e7 Update copyright header check to cover all files
generated files and a list of excluded files are skipped.
Others are fixed to include the missing header.

BUG=b:246579983
TEST=./tools/health-check --all copyright_header

Change-Id: I13e9bf79df18789f1ed4b83fc47c0c2e080d70a8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3894240
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-13 18:41:29 +00:00
Dennis Kempin
2d1a214d38 tools/test_target: Fix mingw64 usage
You can now use:

./tools/test_target --build-target=mingw64 set host && source .envrc

to set up the local environment to build/test via wine64.

BUG=b:241495641
TEST=see above

Change-Id: I498d4fda4cb84c2b8326e91f10b42cf33d8c8c41
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3885375
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-09-08 20:38:19 +00:00
Dennis Kempin
f1e3c34521 common.py: Nit fixes to very-verbose output
Fix stderr output and force single threaded execution to preserve
proper output.

BUG=None
TEST=./tools/health-check -vv --all

Change-Id: I3095e1fcd4cb6abb1852d3fcc3170d5f60141dd5
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3884296
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-09-08 17:34:26 +00:00
Dennis Kempin
f875e532a3 Properly handle colors in subprocesses
We have to manually specify color flags when calling processes. Added a
few utility functions to specify those.

BUG=b:243079837
TEST=./tools/health-check --color=always -v
./tools/health-check --color=never -v

Change-Id: I9095a888bbff264d15d3dbba41a2800155aa770f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3880829
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-08 17:08:59 +00:00
Dennis Kempin
e183aaf26a common.py: Command cleanup
In preparation for further changes this CL clarifies the
Command API a little:

- Clear distinction between foreground and hidden usage.
- Clarified --verbose and --very-verbose flags.
- Clarified builder API by adding with_ prefix.
- Clarified behavior of quiet
- More detailed --very-verbose prints

BUG=b:243079837
TEST=./tools/health-check --all

Change-Id: Ib3d1671a018010c50ee6fae9cd7e9cef0ac8b515
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3880964
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-08 17:08:55 +00:00
Daniel Verkamp
4e36e8a601 rust-toolchain: update to Rust 1.62
This matches the recent ChromeOS SDK update.

BUG=b:243677117
TEST=tools/dev_container tools/clippy

Change-Id: I5de34abfe0985b5f41ac666ce3e93adbb628d790
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3854975
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
2022-09-06 17:36:34 +00:00
Dennis Kempin
232e17c6dd Add tests for ./tools/cl
Makes a temporary copy of the git repo for each test so we can
modify the repo for testing purposes.
It's not the fastest test (~12s), but is only run when python files
have been modified. In contrast to other developer tooling, tools/cl
is not used by Luci, so it needs a dedicated test.

BUG=b:244185215
TEST=./tools/health-check

Change-Id: I06c90a580aa8ed0fa267a41ca40895710121767f
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3866692
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-09-06 17:14:34 +00:00
Keiichi Watanabe
53e745078a vmm_vhost: Enable health-check for third_party/vmm_vhost
Make tools/health-check cover third_party/vmm_vhost, as we are
maintaining vmm_vhost by ourselves and we don't pull the upstream
changes so frequently unlike other third_party components.

This CL includes:
* updates in scripts under /tools, and
* the auto-gerated changes by `./tools/fmt --nightly`

BUG=b:239937122
TEST=cargo check

Change-Id: I12956a60bb24764ffb541261c7fb3f09eb974dd8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3863043
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
2022-09-01 15:05:31 +00:00
Zihan Chen
9c3270b08c dev_container: Fix /scratch permission
Correct permission of /scratch to allow crosvmdev to modify its
content after UID & GID changed to match host after container first
start.

TEST=tools/dev_container --stop ; tools/dev_container tools/run_tests
--target=vm:aarch64 --build-only

Change-Id: I8872c433ae903536750a4623f0d149b2875deba2
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3855471
Commit-Queue: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Zihan Chen <zihanchen@google.com>
2022-08-25 18:34:56 +00:00
Zihan Chen
483345c4c0 dev_container: Use host UID & GID for user in container
Container shell now defaults to user `crosvmdev` which will have
same UID & GID as outside user. VS Code Remote Containers extension
has the same behavior when loading config from devcontainer.json.

TEST=all tests pass in container, new files in container have same
permission as files created outside.
FIXED=b:239476013

Change-Id: Ib3c696509e9ea45edd5f02ff025d9477576ec765
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3814094
Auto-Submit: Zihan Chen <zihanchen@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: Zihan Chen <zihanchen@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-08-23 23:16:09 +00:00
Alexandre Courbot
d19ae375b5 crosvm: disable ffmpeg decoder from default builds
We are about to require ffmpeg >= 5.0 in the ffmpeg decoder backend, but
the version on Debian Bullseye (used in our container and VM images) is
still 4.3. Temporarily disable building the ffmpeg crate and video
decoder by default until we use a system image that includes ffmpeg >=
5.0.

BUG=None
TEST=./tools/dev_container ./tools/run_tests
TEST=./tools/dev_container ./tools/run_tests --target=vm:aarch64
TEST=./tools/dev_container ./tools/run_tests --target=vm:aarch64 --build-target armhf

Change-Id: Idf96862d3ea9bea597701fdee8145e12a0753a3e
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3835500
Tested-by: Alexandre Courbot <acourbot@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
2022-08-23 01:47:48 +00:00
Dennis Kempin
1e16dc6a8f tools/run_tests: Add --cov option
The option will generate an lcov file in the standard name lcov.info
and print a coverage report after running tests.

The lcov.info file is understood by many IDE plugins, e.g. "Coverage
Gutters" for vscode.

BUG=b:239255082
TEST=./tools/run_tests --cov

Change-Id: I475c62ada73f100a984ad863e511083c69807aff
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3840956
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-08-19 18:05:35 +00:00
Dennis Kempin
3b60f0ca33 reland: test_runner: Support multiple profiles per test
Tests may spawn multiple processes (e.g. for running crosvm in
integration_tests). We want those to generate a separate profile
file to collect them as well.

This should add coverage from integration tests into the coverage
data.

Fixed coverage collection with repeated test runs

BUG=b:239255082
TEST=./tools/run_tests --generate-lcov coverage.lcov --repeat 2

Change-Id: I63e812620b79182a5be7e981b8f524b36b596ce2
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3840484
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-08-19 17:51:37 +00:00
Dennis Kempin
39b125eb72 Revert "test_runner: Support multiple profiles per test"
This reverts commit 9ec4afc0ae.

Reason for revert: Breaks post-submit

https://ci.chromium.org/ui/p/crosvm/builders/ci/linux_x86_64/b8805461167965542065/overview

Original change's description:
> test_runner: Support multiple profiles per test
>
> Tests may spawn multiple processes (e.g. for running crosvm in
> integration_tests). We want those to generate a separate profile
> file to collect them as well.
>
> This should add coverage from integration tests into the coverage
> data.
>
> BUG=b:239255082
> TEST=./tools/run_tests --generate-lcov coverage.lcov
>
> Change-Id: Ic6c6b0801b676c96c4692069c1cd6111edea6fc4
> Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3840311
> Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>
> Tested-by: Dennis Kempin <denniskempin@google.com>

Bug: b:239255082
Change-Id: I958792f86a5d5bc24910c057c61242fe405db20d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3840169
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
2022-08-18 21:21:55 +00:00
Dennis Kempin
9ec4afc0ae test_runner: Support multiple profiles per test
Tests may spawn multiple processes (e.g. for running crosvm in
integration_tests). We want those to generate a separate profile
file to collect them as well.

This should add coverage from integration tests into the coverage
data.

BUG=b:239255082
TEST=./tools/run_tests --generate-lcov coverage.lcov

Change-Id: Ic6c6b0801b676c96c4692069c1cd6111edea6fc4
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3840311
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-08-18 18:42:17 +00:00
Dennis Kempin
67961f3d33 tools/impl/common.py: Remove __future__ import
The __future__ import has to be the first line, but will fail
if we are not using at least Python 3.7.
All it changes is the evaluation time of type annotations, so we
have to wrap some names into strings.

BUG=None
TEST=./tools/cl

Change-Id: Ib880fb29fc7a426622d03a996354b13639656c71
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3833640
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Paramjit Oberoi <psoberoi@google.com>
2022-08-18 00:18:27 +00:00
Dennis Kempin
7a4d4d6b2a health-check: Add check for newline at ends of files
Fixes a couple of files that were missing them.

BUG=b:242605601
TEST=./tools/health-check --fix

Change-Id: I620d6a939cb824e014002152584aacfc5dfdf7e8
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3835648
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-08-18 00:17:04 +00:00
Dennis Kempin
38b9268cb8 health-check: Add infra checks
Adds presubmit checks for both lucicfg files and python recipes.

Both can "fix" issues by re-generating configs / test expectations.

This adds depot_tools as a submodule to crosvm, which contains the
required tools for luci.

Does not validate the infra config. Unfortunately this is only
possible with luci authentication which we do not have inside
Docker.

BUG=b:242605601
TEST=./tools/health-check with changes to config and recipes.

Change-Id: I1bf18ebac698e44df7d6a0d6c8e9c26bcfde364b
Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3832783
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-08-18 00:17:04 +00:00