This will clean up the luci log page and failures in any of these
steps will be marked as infra failures.
BUG=None
TEST=None
Change-Id: I7bce6ba3eb34c1a68d7ca8d21d6b5f4e332de17c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739375
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Refactors the crosvm recipe API a little to provide 3 different
environments for builds: Just the source, building with containers
and building on the host.
For building on the host, we will install rustup-init via CIPD and
then use rustup to install the required rust version.
BUG=b:233914170
TEST=./recipes.py run build_windows
Change-Id: I12ef22f286af584edeb02beed4d231565b698099
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3718900
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Updates run_tests to use cargo style target triples for specifying
build targets. A simple 'aarch64' or 'armhf' was nice while we just
had linux builds. We now are looking at windows and possibly
different toolchain options (e.g. msvc vs gnu), so our old system
was getting confusing and inconsistent.
We used to have some special handling for adding wrappers to test
runs for emulation (e.g. wine, qemu). That logic has been moved
into TestTarget which now contains not just where to run the test
but also how.
Supported are armhf/aarch64 qemu as well as wine64.
The CLI has been updated to match and now uses the build-target
argument instead of arch.
The following combinations have been tested (though not all
combinations actually pass all tests, which is a separate issue).
./tools/run_tests
./tools/run_tests --target=host --build-target=x86_64-unknown-linux-gnu
./tools/run_tests --target=host --build-target=armhf
./tools/run_tests --target=host --build-target=aarch64
./tools/run_tests --target=host --build-target=mingw64
./tools/run_tests --target=vm:aarch64
./tools/run_tests --target=vm:aarch64 --build-target=armhf
BUG=b:233914170
TEST=See above
Change-Id: Ic6dbb5b39788e2573714606d3bb0e7c712032d91
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739240
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This will enable a luci build on windows as post-submit while
we test the build_windows recipe.
BUG=b:233914170
TEST=lucicfg validate main.star
Change-Id: I46ba9309f5dbd6077c73d42bbaafe8986ab18af3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3718901
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The Infra console view is replaced with a list view, since
the builds are no longer just post submit builds.
BUG=b:233913643
TEST=lucicfg validate main.star
Change-Id: Ia1ccc75251ccdb7ae567418235e2bf8b7d6aa16a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3692687
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
To enable the merge bot on luci we had to solve a couple of problems:
- We cannot use http cookies for auth, so added gcloud auth into
merge_bot.
- Switch back to original HEAD after running merge bot. Otherwise
the version of the merge_bot script can change between invocations.
- Do not soft reset when checking out crosvm source.
- For less invasive testing, also added a few more exceptions when
using MERGE_BOT_TEST so it won't email, etc.
- Rename to update_chromeos_merges, as it is more fitting to what
the bot is doing
BUG=b:233913643
TEST=https://ci.chromium.org/swarming/task/5b58ed4497fda510
Change-Id: Iaa9b4821b70a1e90d19637d01856196bd7852ed5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3692686
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
This builder re-creates ci/build_merge_into_chromeos and just calls
the merge_bot script.
Eventually this could be better integrated with Luci UI. For now,
this is feature parity with Kokoro.
BUG=b:233913643
TEST=recipe.py run merge_into_chromeos
Change-Id: Ia5de775cad943687be654c3a054f1150ba9adbb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3673786
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Recipes can be tested with unit tests, run locally or run on a build
bot. The documentation shows how to do this.
BUG=None
TEST=None
Change-Id: Iec139de5c38d45765c9da0d95e8ac076c5df1dfd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3682200
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Kokoro would maintain a local main branch in the local checkout,
so pushing --all will push the latest HEAD of main.
Luci will instead put us on a detached HEAD, with the local
main branch unmaintained. So we have been pushing the same old
version of crosvm.
Update the force push to submit HEAD to the remote main.
BUG=b:233913820
TEST=./recipes.py run push_to_github (requires authentication with
github to run)
Change-Id: I8282498382968768d66d9c807b9e86d191977beb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3681989
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Some recent changes introduced a bug that breaks reusage of
containers. Instead of trying to restart a stopped container,
we delete it instead and start a fresh one.
The code was refactored to allow for testing. The tests run
against the real docker service and are run as part of the health
checks in kokoro / luci.
This CL also gets rid of the custon command line parsing since
luci can now run it with argh available.
BUG=b:234402839
TEST=./tools/dev_containes --self-test [--podman]
Change-Id: Id5eaf0ea83ff07433f8f57cb652b5c393dcb9da3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3681399
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The builder runs as a post-submit and will force push changes to github.
Authentication is done via an access token stored in gcloud secrets
manager.
For this to work in the bot pool, the bots need to be reconfigured to
receive the scope needed for access to the secrets API as done in
https://crrev.com/i/4782519
Once this bot is working, the kokoro one can be turned off. Since we
mirror from platform/crosvm to the new repo, luci is triggered for all
new changes.
BUG=b:233913820
TEST=./recipe.py run push_to_github
Change-Id: I45478a076b87767ec10f0d0148c0713da7656264
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3671131
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
The board name is amd64-generic, not amd64_generic.
BUG=b:233913797
TEST=None
Change-Id: I11a0ba7cda84614d757d58563d49cb476c26e47f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3680778
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Without, gerrit will not let us submit code and we won't be able to use
repo.
BUG=b:233913797
TEST=recipes.py test run
Change-Id: I59dc6633c716288094e02f38d613343e69c8c3e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674004
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
- Use vpython to call crovm tooling (This provides a predictable python
environment and allow us to use argh).
- Add build_context() to provide a simple API that readies source and
container.
- Nest preparation steps to reduce noise on builder page.
BUG=b:233913643
TEST=recipes.py test run
Change-Id: I6800e55f7311b32c6ef61918d13795af1fb3b588
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674217
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
The builders do not have depot_tools installed globally, but
the recipes checkout contains them, so look up the proper
paths to the tools.
BUG=b:233913797
TEST=./recipes.py run build_chromeos
Change-Id: I5d50871fa0ccee769704b339a20d3406d23e0510
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674216
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
This re-creates the Kokoro equivalent crosvm/ci/build-chromeos.sh.
It is a very inefficent builder that does a fresh checkout of
chromeos for every build.
We may want to consider using some of the chromiumos recipes
for caching the repo or sdk between builds.
BUG=b:233230344
TEST=./recipes.py test run
./recipes.py run build_chromeos
Change-Id: I5d545bae4817cae5b79c5d65901081d9b1391466
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3664842
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Adds a builder for both pre- and post-submit, executes the newly
added health_check recipe.
BUG=b:233913455
TEST=lucicfg validate main.star
Change-Id: I69e9738a02bb298d3b2556058d2df1ee9979bda4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3669815
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This allows us to run multiple steps with the same intermediate
build data.
To make sure we start with a clean slate, any existing containers are
deleted at the beginning of each recipe.
BUG=b:233913455
TEST=./recipes.py test run
Change-Id: Ieda684d40bf08937183cba34242360c025724af8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3668813
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
The builders use the existing dev_container to run tests
for the 3 currently supported architectures on linux:
x86_64, aarch64 and armhf.
A new recipe 'build_linux' has been added runs the
dev_container to build and execute tests.
To share code with other upcoming recipes, some of the
recipe code has been extracted into a shared crosvm
module.
Basic tests for the recipes have been added.
BUG=b:233230344
TEST=./recipes.py test run
./recipes.py run build_linux
Change-Id: I17ecb25c0a0eabdce56537831454ac22d4dc7021
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3654196
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Turn off crosvm builders in Luci temporarily since the cloud storage
bucket in crosvm-infra doesn't grant proper access for logdog.
Bug:1300370
Change-Id: Ie91b33d10c1a9e6e155961be4cfb8c7a406fcef5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3636269
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Yuanjun Huang <yuanjunh@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
The recipe follows luci onboarding instructions to verify configuration.
It currently just checks out the gerrit change provided as input.
BUG=1300370
TEST=Basic test included
Change-Id: I249ce2124f4b161465d7603155cbe25a85d2551a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3626223
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Yuanjun Huang <yuanjunh@google.com>
Fix validation errors in luci-scheduler.cfg:
1. (acl_sets): aclSet "ci" has no entries
2. (job / Example Builder / task): legacy v1 bucket names like
"luci.crosvm.ci" are no longer allowed, use "ci" instead
3. (job / Example Builder / acl_sets): referencing AclSet "ci" which
doesn't exist
4. (job / Example Builder): Job or Trigger must have OWNER acl set
The setup doc doesn’t mention what the ACL values should be. So I coped
ACL configs from `projects/infra` configs and those can be changed later.
Bug: chromium:1300370
Change-Id: Iffc57e37b62d5048ccff83836bb6dddec9e34c68
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3617164
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Yuanjun Huang <yuanjunh@google.com>
We want to auto-submit trivial rolls and send non-trivial ones
to denniskempin@ for now. This will later be replaced with
a rotation.
BUG=None
TEST=None
Change-Id: I57c4707262cc3a09a70dff9013b9e1c8c434eef0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3615514
Reviewed-by: Yuanjun Huang <yuanjunh@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
