Commit graph

16 commits

Author SHA1 Message Date
Dennis Kempin
e3db2b5638 Add crosvm_linux_x86_64_direct builder
Similar to the one we added to kokoro, run tests with --crosvm-direct.

BUG=b:238234098
TEST=recipe tests included

Change-Id: I2b03ca02fbfb577fd5cebf346eec468ebd625626
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3749937
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-07-07 17:48:49 +00:00
Dennis Kempin
070cc87986 infra: Nest preparation steps and mark them as infra steps
This will clean up the luci log page and failures in any of these
steps will be marked as infra failures.

BUG=None
TEST=None

Change-Id: I7bce6ba3eb34c1a68d7ca8d21d6b5f4e332de17c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739375
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-07-01 21:34:07 +00:00
Dennis Kempin
323d267a8c infra: Add build_windows recipe
Refactors the crosvm recipe API a little to provide 3 different
environments for builds: Just the source, building with containers
and building on the host.

For building on the host, we will install rustup-init via CIPD and
then use rustup to install the required rust version.

BUG=b:233914170
TEST=./recipes.py run build_windows

Change-Id: I12ef22f286af584edeb02beed4d231565b698099
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3718900
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-07-01 20:11:59 +00:00
Dennis Kempin
767e094fb8 tools/run_tests: Use triples for arch
Updates run_tests to use cargo style target triples for specifying
build targets. A simple 'aarch64' or 'armhf' was nice while we just
had linux builds. We now are looking at windows and possibly
different toolchain options (e.g. msvc vs gnu), so our old system
was getting confusing and inconsistent.

We used to have some special handling for adding wrappers to test
runs for emulation (e.g. wine, qemu). That logic has been moved
into TestTarget which now contains not just where to run the test
but also how.

Supported are armhf/aarch64 qemu as well as wine64.

The CLI has been updated to match and now uses the build-target
argument instead of arch.

The following combinations have been tested (though not all
combinations actually pass all tests, which is a separate issue).

./tools/run_tests
./tools/run_tests --target=host --build-target=x86_64-unknown-linux-gnu
./tools/run_tests --target=host --build-target=armhf
./tools/run_tests --target=host --build-target=aarch64
./tools/run_tests --target=host --build-target=mingw64
./tools/run_tests --target=vm:aarch64
./tools/run_tests --target=vm:aarch64 --build-target=armhf

BUG=b:233914170
TEST=See above

Change-Id: Ic6dbb5b39788e2573714606d3bb0e7c712032d91
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3739240
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-07-01 19:16:59 +00:00
Dennis Kempin
5156976b7f merge_bot: Enable on luci
To enable the merge bot on luci we had to solve a couple of problems:

- We cannot use http cookies for auth, so added gcloud auth into
  merge_bot.
- Switch back to original HEAD after running merge bot. Otherwise
  the version of the merge_bot script can change between invocations.
- Do not soft reset when checking out crosvm source.
- For less invasive testing, also added a few more exceptions when
  using MERGE_BOT_TEST so it won't email, etc.
- Rename to update_chromeos_merges, as it is more fitting to what
  the bot is doing

BUG=b:233913643
TEST=https://ci.chromium.org/swarming/task/5b58ed4497fda510

Change-Id: Iaa9b4821b70a1e90d19637d01856196bd7852ed5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3692686
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-08 17:20:27 +00:00
Dennis Kempin
9cf56a869e infra: Add merge_into_chromeos builder
This builder re-creates ci/build_merge_into_chromeos and just calls
the merge_bot script.

Eventually this could be better integrated with Luci UI. For now,
this is feature parity with Kokoro.

BUG=b:233913643
TEST=recipe.py run merge_into_chromeos

Change-Id: Ia5de775cad943687be654c3a054f1150ba9adbb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3673786
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-06-03 19:18:06 +00:00
Dennis Kempin
dd4b8f1676 infra: Add push-to-github builder
The builder runs as a post-submit and will force push changes to github.
Authentication is done via an access token stored in gcloud secrets
manager.

For this to work in the bot pool, the bots need to be reconfigured to
receive the scope needed for access to the secrets API as done in
https://crrev.com/i/4782519

Once this bot is working, the kokoro one can be turned off. Since we
mirror from platform/crosvm to the new repo, luci is triggered for all
new changes.

BUG=b:233913820
TEST=./recipe.py run push_to_github

Change-Id: I45478a076b87767ec10f0d0148c0713da7656264
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3671131
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2022-05-31 21:26:19 +00:00
Dennis Kempin
9c313915f5 infra: Set git name/email on all builders
Without, gerrit will not let us submit code and we won't be able to use
repo.

BUG=b:233913797
TEST=recipes.py test run

Change-Id: I59dc6633c716288094e02f38d613343e69c8c3e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674004
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-05-28 01:04:01 +00:00
Dennis Kempin
de651d0357 infra: Small process improvements
- Use vpython to call crovm tooling (This provides a predictable python
  environment and allow us to use argh).
- Add build_context() to provide a simple API that readies source and
  container.
- Nest preparation steps to reduce noise on builder page.

BUG=b:233913643
TEST=recipes.py test run

Change-Id: I6800e55f7311b32c6ef61918d13795af1fb3b588
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674217
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Anton Romanov <romanton@google.com>
2022-05-27 20:59:12 +00:00
Dennis Kempin
507ac947ae infra: Use depot_tools for repo and cros_sdk
The builders do not have depot_tools installed globally, but
the recipes checkout contains them, so look up the proper
paths to the tools.

BUG=b:233913797
TEST=./recipes.py run build_chromeos

Change-Id: I5d50871fa0ccee769704b339a20d3406d23e0510
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3674216
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
2022-05-27 19:01:29 +00:00
Dennis Kempin
b360cd9fa9 infra: Add ChromeOS post-submit builder
This re-creates the Kokoro equivalent crosvm/ci/build-chromeos.sh.

It is a very inefficent builder that does a fresh checkout of
chromeos for every build.

We may want to consider using some of the chromiumos recipes
for caching the repo or sdk between builds.

BUG=b:233230344
TEST=./recipes.py test run
./recipes.py run build_chromeos

Change-Id: I5d545bae4817cae5b79c5d65901081d9b1391466
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3664842
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-05-26 22:03:51 +00:00
Dennis Kempin
cb99b70b1d infra: Add health check builder
The builder calls ./tools/health-check for fmt/clippy/python checks.

BUG=b:233913455
TEST=recipes.py run health_check

Change-Id: Ia4b8298ef921b33a687d4e64956fc1f017649e7f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3668814
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-05-26 17:47:46 +00:00
Dennis Kempin
bcbf2cb678 infra: Preserve container between steps
This allows us to run multiple steps with the same intermediate
build data.
To make sure we start with a clean slate, any existing containers are
deleted at the beginning of each recipe.

BUG=b:233913455
TEST=./recipes.py test run

Change-Id: Ieda684d40bf08937183cba34242360c025724af8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3668813
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-05-26 17:47:46 +00:00
Dennis Kempin
0948bc8d3a infra: Initial setup of linux pre/post-submit builders
The builders use the existing dev_container to run tests
for the 3 currently supported architectures on linux:
x86_64, aarch64 and armhf.

A new recipe 'build_linux' has been added runs the
dev_container to build and execute tests.

To share code with other upcoming recipes, some of the
recipe code has been extracted into a shared crosvm
module.

Basic tests for the recipes have been added.

BUG=b:233230344
TEST=./recipes.py test run
./recipes.py run build_linux

Change-Id: I17ecb25c0a0eabdce56537831454ac22d4dc7021
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3654196
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2022-05-24 23:13:13 +00:00
Dennis Kempin
6b2b2184fd infra: Add example verify_cl recipe
The recipe follows luci onboarding instructions to verify configuration.

It currently just checks out the gerrit change provided as input.

BUG=1300370
TEST=Basic test included

Change-Id: I249ce2124f4b161465d7603155cbe25a85d2551a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3626223
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Yuanjun Huang <yuanjunh@google.com>
2022-05-04 21:40:49 +00:00
Dennis Kempin
793004384d infra: Add recipes and example builder
Following the onboarding instructions. If everything works we should
have a builder at:
https://ci.chromium.org/p/crosvm/builders/ci/Example%20Builder

BUG=chromium:1300370
TEST=None

Change-Id: I98eeaad6ccffd228fdee116e664c9d2760708e24
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3500817
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2022-04-26 19:26:47 +00:00