Commit graph

18 commits

Author SHA1 Message Date
Keiichi Watanabe
610158aa55 ci: kokoro: push-to-github: Use git push --all instead of --mirror
Unlike git push --mirror, git push --all won't delete remote branch so
that gh-pages branch won't be removed by kokoro, I think.

BUG=b:199692764
TEST=none

Change-Id: I2a5ce2f59e966775d24c25b996a1091af1d6efc9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3234557
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
2021-10-20 16:06:04 +00:00
Dennis Kempin
8a1277de1a Kokoro: Rebase changes to ToT before testing
By default Kokoro will use changes as they come from gerrit, which may
have an outdated parent. We want to make sure that we are always testing
against tip of tree so we are confident the presubmit test result will
reflect the postsubmit results.

BUG=b:202275156
TEST=Tested in Kokoro

Change-Id: I2a1d2860a361eee741dc522c9859e32890ff15bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229947
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-10-19 17:10:04 +00:00
Dennis Kempin
76e6fd051b Fix push-to-github Kokoro builder
One more fix. Everything seems to work, but the remote is rejecting
an update of the active main branch. Hopefully the last fix needed.

BUG=b:199692764
TEST=Testing in prod.

Change-Id: I4f341cc374080454317cb3ac29cade59fec6aeee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229958
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-10-18 23:22:27 +00:00
Dennis Kempin
2d50ccb4a7 Fix push-to-github kokoro build
We were in the wrong directory.

BUG=b:199692764
TEST=Testing in prod...

Change-Id: Idbc60f7f6084109220186e56cd7fad2e6553da9a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229950
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-10-18 22:00:22 +00:00
Dennis Kempin
49c169a410 Add push-to-github kokoro config and script
This is hard to test. I verified that kokoro can access
the github token and use it to access github by adding
a git clone into the x86 presubmit build file in patchset 3.

But we can only really test the full process by submitting this
CL and have it pass through the continuous builder.

BUG=b:199692764
TEST=Needs to be tested in prod...

Change-Id: Ifdcd385552c61ecea75a43fb71b08950e8b036ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229941
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-10-18 21:37:58 +00:00
Dennis Kempin
d1dbefc9dc Integrate new container and test runner with Kokoro
This replaces the existing tooling at ./ci/ with the new tooling
at ./tools.

BUG=b:199951064
TEST=./ci/kokoro/simulate_all

Change-Id: Id0296d5a46ca0f3eb0f655d85d1aed801ee3b434
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3226275
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-10-18 20:10:42 +00:00
Dennis Kempin
e9aa5f9598 Fix ./ci/kokoro/simulate
Somehow, after rsyncing the whole directory, git will no longer work in the
copy. So instead, we are now using 'git clone'.
This changes the behavior to not include local workspace modifications in
kokoro simulations. Which is ok, we have test_all for that and the behavior
will more closely match what's happening in kokoro presubmits.

BUG=None
TEST=./ci/simulate_all

Change-Id: I439b9eadcac65d99782e2b0eb869519abb2ada37
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3152425
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-09-15 17:06:48 +00:00
Dennis Kempin
3b3545b2c3 Upgrade ci containers to bullseye and clean up
We no longer need to build dependencies from source and provide them
via pkg-config, this simplifies much of our builder container setup.

As debian bullseye got promoted to stable, we can now also simplify
further by just using bullseye stable instead of mixing stable and
testing packages.

BUG=b:181359683
TEST=./test_all

Change-Id: I2ce61992d5cfe6eb5dc3f0ec61920dcc5455ca40
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3141772
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-09-15 17:06:46 +00:00
Dennis Kempin
42b825f740 Switch to submodules based workflow
Instead of requiring crosvm to be checked out via the chromeos manifest
to access dependencies in the ChromeOS monorepo, this change adds
git submodules to third_party/.

The CI scripts and Cargo.toml are updated to use the new paths.

BUG=b:194336213
TEST=git clone --recursive https://chromium.googlesource.com/chromiumos/platform/crosvm
     cd crosvm
     cargo test
     ./test_all
     ./ci/kokoro/simulate_all

Change-Id: I9859d18176e21909ac3a140976fbd67cc14129bb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3049003
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-08-05 18:32:32 +00:00
Dennis Kempin
98d2f0294b Kokoro: Always build against ToT instead of a fixed manifest
This will introduce a little instability into our test infrastructure
since tests will no longer be hermetic, but build against the latest
ToT version of our chromeos dependencies.

It will however, make our day-to-day workflows a lot easier. A lot of
changes are made across repository boundaries and require
annoying extra steps since Kokoro does not understand Cq-Depend.

BUG=b:185917395
TEST=./ci/kokoro/simulate_all passes

Change-Id: I3c3cae20d418574a3db53cc74d81245b87c228db
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2840057
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-04-21 04:51:34 +00:00
Dennis Kempin
78145acd86 Make Kokoro happy
Update Kokoro manifest and Cargo.lock.

This change does not affect ebuild builds, so it can be safely chumped
to get Kokoro green.

BUG=None
TEST=./ci/kokoro/simulate_all passes

Change-Id: If4e2b849f12d4e223dcfe6dd5bb48e034067b466
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2840055
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-04-20 23:05:26 +00:00
Zach Reizner
d09392e37e uprev rust-toolchain and fix clippy warnings
Some judgement calls were made about unnecessary wrapping. Usually they
would get resolved by removing the wrapping or returning a convenient
error, but the ones that returned results for consistency with other
functions were added to the allow list.

The error handling in the usb code had a lot of unit error types which
is now a clippy lint. This was resolved by either removing the result
entirely or returning a convenient error.

The field_reassign_with_default lint is faulty and was added to the list
of supressions. This affected virtio-wayland code.

BUG=b:179277332
TEST=cargo clippy with rustc 1.50+

Change-Id: Ie812cdeaf7c42f4f2b47b1dc87f05a7c87a60f8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2757510
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-22 21:41:07 +00:00
Keiichi Watanabe
5dc0f60d44 kokoro: Add rust-vmm/vhost in manifest.xml
BUG=b:181227406
TEST=ci/kokoro/simulate_all

Change-Id: I65065963daa9e96457ed9a64f01e34ad93bd8667
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2762488
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-17 19:12:08 +00:00
Dennis Kempin
f248a8ec2b Fix Kokoro build with manifest uprev
Kokoro was failing since http://crrev.com/c/2744968 depends on recent
cras changes that were not included in the current Kokoro manifest.

BUG=None
TEST=./ci/kokoro/simulate_all passes

Change-Id: I470844af73b9f89ac8cbf9f5fcf8706d01924279
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2746014
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-10 06:16:25 +00:00
Dennis Kempin
3682372764 ci: Upgrade test-vm to debian bullseye
This change updates the test VM to debian bullseye, primarily to enable
io_uring features that are not available in the 4.16 kernel shipping
with debian buster.

We are not updating the builder containers to bullseye, as newer
versions of qemu fail to start on kokoro. See b/181359683

Since it was needed for debugging this CL, it includes a feature to
debug kokoro builders via SSH. It can be enabled by setting
DEBUG_SSH_KEY from the fusion2 UI.

BUG=b:178228512
TEST=./test_all passes

Change-Id: I22f1678577183a0aace80245a87c31907221cc09
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2721062
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-10 03:37:00 +00:00
Dennis Kempin
4d2253b529 Kokoro: Extensive polishing and bugfixing
This CL fixes some of the issues that prevented the
test system from running on other workstations.

- Builders run by Kokoro will not use a scratch mount.
  It's not needed, and will cause issues if stored on
  /tmp with noexec set.
- Running podman with label=disable to prevent selinux
  issues.

The run_tests implementation has been moved to a separate
file and updated with parsing of cargo test output. This
allows simpler test output and integration with the
test results UI in sponge.

The sponge test UI can make it much easier to see which
tests failed, and to find the log of that test case.

This CL also includes an ./ci/kokoro/uprev script to
uprev the manifest versions. And runs the uprev.

BUG=b:174861002
TEST=Tested by forcing a kokoro build with this CL

Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-10 22:04:43 +00:00
Dennis Kempin
26cc51e588 Make Kokoro v2 happy with new config
Fixes some nits that cause kokoro builds to fail and updates
the Kokoro configs to match the new Kokoro-side configs
submitted in cl/355894530.

With this CL Kokoro v2 should be passing and can be enabled as
a presubmit.

BUG=b:179289824
TEST=Kokoro passes

Change-Id: I8c9259407bf1eb932b05376b35cb88d2e7d6058c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2678792
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-02-06 05:26:04 +00:00
Dennis Kempin
0dbb9808a6 Add Kokoro for ci/builders
Adds the crosvm-side infrastructure to build and test
in kokoro.
There is a build script for testing on x86, aarch64
and a separte script for analysis (clippy, fmt).
These will run in parallel on Kokoro. To test the
scripts locally, a simulate script is provided.

Runtime on my workstation:
- aarch64: 10m
- x86: 2:30m
- analysis: 1:40m

BUG=b:177951955
TEST=./ci/kokoro/simulate_all

Change-Id: I2f666ec768e6c3391a258dc7f0cbd999ad9b2fb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2654413
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
2021-02-05 03:01:55 +00:00