- Use vpython to call crovm tooling (This provides a predictable python
environment and allow us to use argh).
- Add build_context() to provide a simple API that readies source and
container.
- Nest preparation steps to reduce noise on builder page.
BUG=b:233913643
TEST=recipes.py test run
Change-Id: I6800e55f7311b32c6ef61918d13795af1fb3b588
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674217
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
This just creates unnecessary noise. Just create a single dry run
merge commit each time.
BUG=b:227224259
TEST=None
Change-Id: I54385fd80f01771bb83e02f2b48e604ad2d68d1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3673021
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Otherwise we may exit thinking we are trying to run an interactive
command in a non-interactive terminal.
BUG=None
TEST=./tools/dev_container --stop </dev/null
Change-Id: Ie34cf662b82090b100533306572552c5a550211e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3671132
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
The default is still to run all of them, but this will allow us to run
checks in separate luci steps to provide an easier to read result.
BUG=b:233913455
TEST=./tools/health-check python fmt
Change-Id: Iddc803f8f423db36ece53a13acfe564560b789a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3668812
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
We run a neverending command in the container to keep it alive in the
background.
By default, this would be bash, which does the job when --tty is
specified. But we no longer do that all the time.
Instead we can use sleep, which does not rely on such tricks.
BUG=b:233913455
TEST=./tools/dev_container true
Change-Id: I2d04aabbef6b670daf6b94fdc2c65c7bd759c61b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3669682
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
They have been dropped with the recent refactoring to not use argh.
BUG=None
TEST=./tools/dev_container -v
Change-Id: Ifa65b14d316e27dbb5cef9714fbd3adf110b79e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3664051
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This fixes little annoyances where fmt or other tools would try to run
on files not part of version control (e.g. local editor configs).
BUG=b:227224259
TEST=./tools/fmt -v
Change-Id: Iddff90ec95a9e28cd80ac3aece26202e8a42e9d3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3662022
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
By passing main(..., podman=args.podman), we mask the function's default
value for that parameter which is False when --podman isn't passed. This
doesn't match what 34811a88ee ("dev_container: Add --podman command
line parameter") says about the flag:
The parameter [--podman] defaults to true if docker is not installed
Instead, use the computed default for the argument itself.
BUG=b:233346153
TEST=tools/dev_container bash -c 'echo Hello'
Change-Id: I2e5f9855853279a8ceeb9347931d59fd334f0004
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3655968
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Pierre-Clément Tosi <ptosi@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This script is naturally used outside of the container and we
want to avoid having to install dependencies or special python
versions to run it.
This CL reduces the python requirement from 3.9 to 3.8 and
allows the script to run without any dependencies.
BUG=b:233230344
TEST=pyenv global 3.8.15
./tools/dev_container
Change-Id: Ieeb868f7fb275a8e6e39e577b5e984895307765e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657816
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
Sometimes we are left with an existing container that has been stopped.
Usually happens after reboots.
Make sure we start the container again before trying to execute.
BUG=None
TEST=docker stop $USER_dev_container
./tools/dev_container
Change-Id: Ib68091a09546cfefa452ddb8e6f268e495f2167c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657815
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
The parameter defaults to true if docker is not installed. This also
resolves previous issues with resolving which docker command to use.
A warning is added when executing dev_container with podman.
BUG=b:233346153
TEST=./tools/dev_container --podman
Change-Id: I1d3dad6a657c0273c82634e409a0991245f56f4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657814
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This fixes problems with freezing when running luci recipes locally.
An interactive process can still be forced by flag -i or --interactive.
BUG=b:233230344
TEST=cd infra && ./recipes.py run build_linux
Change-Id: I261f0ffddcc3795e17bcbe023fa920f48769fdb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657813
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
This will be run when using repo upload from a chromiumos checkout
and reminds developers that Cq-Depend cannot be used with crosvm.
BUG=b:226975263
TEST=./tools/chromeos/check_cq_depend
Change-Id: Ib9cafaba181cb2103838369123e6c1288e653486
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3651674
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
crosvm-fuzz only contains fuzzer binaries, which don't have any
documentable items and clutter up the list of crates.
BUG=None
TEST=tools/cargo-doc
Change-Id: I0a8a5a61977643920d66a0f2662a1050a66192a0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3630425
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
This helps with integration of `tools/clippy` in `rust-analyzer` and
potentially other IDEs which can parse the json output better than the
text output.
TEST=Ran it with `rust-analyzer` in VSCode.
Change-Id: Iceb2748994cbbacf48dc473a9ad06eec567beb0a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3630700
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vaibhav Nagarnaik <vnagarnaik@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This reverts commit 15a4ee3200.
Reason for revert: Should pass now
Original change's description:
> Do not run cros_async tests on armhf
>
> The test has been flaking with segfaults occasionally.
>
> BUG=None
> TEST=presubmit
>
> Change-Id: Ifbfee7e7ed5de91b7bdf9b25db7983139951f81f
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606751
> Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
> Tested-by: kokoro <noreply+kokoro@google.com>
> Commit-Queue: Dennis Kempin <denniskempin@google.com>
Bug: None
Change-Id: I85d0885ce9f9fa3d46dd86536a5a20a71ffbbabb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3626010
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
The sys_util crate has been merged into base, but a few places still
refer to the old name. Fix or remove them as appropriate.
BUG=b:227226222
TEST=tools/presubmit --all
Change-Id: Icf9b57aff672b7c1afec768c9694e059f0f9a43d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3621205
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Fixes cases when we use target vm:aarch64 but arch armhf and test_runner
picks up tests for aarch64
BUG=none
TEST=run and made sure armhf-disabled tests are not picked up
Change-Id: I4d0734bb6320fc698bf16adc59de5e03b00a4a68
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3617081
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Anton Romanov <romanton@google.com>
The test has been flaking with segfaults occasionally.
BUG=None
TEST=presubmit
Change-Id: Ifbfee7e7ed5de91b7bdf9b25db7983139951f81f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606751
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
And fix a nit in the existing tests to make it pass.
BUG=b:219965702
TEST=presubmit
Change-Id: I65fb98b04d7042755930dd34913f679fa129c88c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3594248
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Sligthly changes the logic. If we are on GCE, always use the authdaemon
and set up the git user info for the service account.
Outside of GCE just check if a cookie is available.
BUG=None
TEST=Faked out is_gce_instance and ran locally
Change-Id: Icf4c521b650b5d47f395a59695d1f1470621f661
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585016
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
BUG=None
TEST=Ran locally, but cannot test actually running the tool outside of
GCE.
Change-Id: I4483d8ea75fceb3d289a38d2aa3b8c6f85985dfa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3588403
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
This change adds python type and formatting checks and
consolidates code health checks in ./tools/health-check.
Dealing with relative imports in python is tricky, so
we are making ./tools/impl a proper package with no
directly executable files.
Some of the bash shorthands in ./tools had to be converted
to python for this.
To make the new checks pass, we run the formatter and fix
some mypy type checks.
TEST=./tools/health-check
BUG=b:218559722,b:219965702
Change-Id: Ie18d3d6dd2f5a033141e167a6e1aa762791941d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3558592
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This would previously trigger a program not found error.
BUG=None
TEST=Added doctest
Change-Id: I357c44d2b9ce7b2a3240958c5c88008a29164baa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3587885
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Metadata-Flavor is a header and not part of the body.
BUG=None
TEST=Has to be tested in GCE.
Change-Id: I92711a86502d19601a5f2d63e6a653743b5b2614
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3579815
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
The curl version of Kokoro is too old and does not know it.
BUG=None
TEST=testing in prod.
Change-Id: I8e1fe856dcd893944da06f6a9097497a9c320c43
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3584319
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
If the value is not set, treat it as empty.
BUG=None
TEST=Tested by removing the config setting locally.
Change-Id: Ia160285c03bffcf7a332f3891587e7ada2228623
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585860
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
The new merge bot is written in python to allow us to improve the
merge process.
The new process is as follows:
- The merge bot will only create a new merge when the previous
ones have been submitted.
- When the bot creates a new merge, it'll split the merge commits
into smaller changes to make bisection easier.
This allows us to only ever deal with one set of merge commits
instead of piling on new merges day after day.
For dry runs, the new process is:
- The bot will check existing dry runs and abandon them if the CQ
is done.
- The bot will vote Verified+-1 to indicate if the dry run passed.
- If no more dry runs are active, it'll create a new one.
This ensures we always have a dry run in the CQ and can easily see
in gerrit which ones have passed.
The python script will also allow us to add convenience features
for oncall for retrying bratches, etc.
BUG=b:227347397
TEST=The script can be run locally:
./tools/chromeos/merge_bot update-merges
./tools/chromeos/merge_bot update-dry-runs
Change-Id: I51fc0741f5032868ad07f5e926e570e29cc461fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583254
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Pass `--workspace` and `--no-deps` as cargo-doc's arguements.
As a result:
* it won't generate API docs for third-party crates
* it'll generate API docs for our first crates that are not used by the
minimal crosvm build. (e.g. qcow_utils, libvda, etc)
BUG=none
TEST=test on personal GitHub
Change-Id: I8911eb7d830f1cda77d264c1dc19ff948cfa4768
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3578146
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
This is a reland of commit 2b85d4d1e5
Diff from the original CL:
* Changed the return value of `main()` in power_monitor/build.rs
* Ran cargo-check with all features enabled
Original change's description:
> github: Minimize dependencies for document generation
>
> Instead of install full dependencies with `install-deps` in GitHub
> action, minimize dependencies because we use GitHub only for document
> generation.
>
> - Passed `CARGO_DOC` environment when running cargo-doc command so we
> can skip unnecessary build flow when we just want to generate API docs.
> - Added a new script `install-docs-deps` to install only doc-related
> dependencies.
>
> BUG=none
> TEST=test on GitHub Action at my personal repository
>
> Change-Id: Ibe988ab43215e285d946812bdd6c1536ae87b50e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3578144
> Tested-by: kokoro <noreply+kokoro@google.com>
> Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
> Reviewed-by: Anton Romanov <romanton@google.com>
> Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Bug=none
TEST=cargo check --all-features
Change-Id: I77ee6543910e3fe7f69be48f8f965eda3433d4e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3584063
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
This reverts commit 2b85d4d1e5.
Reason for revert: breaks powerd compile
Original change's description:
> github: Minimize dependencies for document generation
>
> Instead of install full dependencies with `install-deps` in GitHub
> action, minimize dependencies because we use GitHub only for document
> generation.
>
> - Passed `CARGO_DOC` environment when running cargo-doc command so we
> can skip unnecessary build flow when we just want to generate API docs.
> - Added a new script `install-docs-deps` to install only doc-related
> dependencies.
>
> BUG=none
> TEST=test on GitHub Action at my personal repository
>
> Change-Id: Ibe988ab43215e285d946812bdd6c1536ae87b50e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3578144
> Tested-by: kokoro <noreply+kokoro@google.com>
> Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
> Reviewed-by: Anton Romanov <romanton@google.com>
> Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Bug: none
Change-Id: Ie16b65235490071ac4e69bf3f09af3ba0c766eab
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3581836
Auto-Submit: Junichi Uekawa <uekawa@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
Instead of install full dependencies with `install-deps` in GitHub
action, minimize dependencies because we use GitHub only for document
generation.
- Passed `CARGO_DOC` environment when running cargo-doc command so we
can skip unnecessary build flow when we just want to generate API docs.
- Added a new script `install-docs-deps` to install only doc-related
dependencies.
BUG=none
TEST=test on GitHub Action at my personal repository
Change-Id: Ibe988ab43215e285d946812bdd6c1536ae87b50e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3578144
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Anton Romanov <romanton@google.com>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
This replaces the bindgen-based bindings with Rust libc crate types and
constants where possible, so our custom bindings are significantly
reduced.
BUG=b:218388029
TEST=tools/presubmit --quick
Change-Id: I7f20ecf5ccd651456f9a23ce7964fdbd0e8064ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3339851
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
As we are going to introduce more media-related crates, reserve the
"media" folder as a placeholder for them, starting with the existing
libvda.
BUG=b:169295147
BUG=b:214478588
TEST=cargo build --features "video-decoder,video-encoder,libvda"
Change-Id: I1b2ec65cbba8b735db3d19845c504546fa1c64ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3565623
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Alexandre Courbot <acourbot@chromium.org>
Also fixes few warning introduced during upstream which linux clippy
run does not catch.
test: py tools\clippy
bug: 226966790
Change-Id: I6979f87be5d6d34727afabafcaccb57631975567
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3553401
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
This allows the bot to account for manually cherry-picked commits.
To make rebases work properly, we had to fix the commit message
trailers to be added in a separate paragraph. This ensures
git inspect-trailers can add and find the Commit-Id trailer.
BUG=b:226975273
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-merge-into-chromeos.sh
Change-Id: If008be1f1c671ef1d79c6f223f10ea0e8f6742b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3561015
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Does couple of things
- move tube under right directory
- cleanup cros_async dependency
- export tube module from base
- enable base windows tests
- add verbose flag to test runner
test: built on window and presubmit
bug: b:213151429
bug: b:213153157
Change-Id: I0d1906d7edcc3352db53325ea7527550eaf00895
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3550296
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
