Use the new common.py utilities to make turn this bash script into
python. The behavior should be the same.
BUG=b:218559770
TEST=./tools/presubmit
Change-Id: Id1412c2076089fd21f280959d6cde1f4cb64e163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3469049
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
To help find missing feature cfg checks, add a build with all features
disabled to the CI scripts.
BUG=None
TEST=tools/presubmit
Change-Id: I98f38ecc3bf90cbeb081a1fafe4f2f540e4fc348
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3437362
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Build config sets timeout to 40 minutes.
helper script relies on choco to install rustup, which is used
to install rust toolchain.
Bug: 213603713
Test: Tested the toolchain setup part of the script in a
codelab repo.
Change-Id: Ibf139bbc4ff6fe6c811652767dd885584414dda6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3421515
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Brandon Amiosci <brandonamiosci@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Does the same thing, but in python. This should allow us to make
this work on windows as well.
BUG=b:216833094
TEST=./simulate.py build-x86_64.sh
Change-Id: If6eb214c2a3185fbb60a636b89cd03ecf28b42e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3425651
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The dry run merge is re-created every day and automatically
set to CQ+1. This allows us to check for bad crosvm changes
while we submit parts of the merge chain.
BUG=b:215741244
TEST=./tools/ci/simulate merge-into-chromeos.sh
Change-Id: I0773a99f9858962e9f25bebff1b233d455d33cdd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3417508
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
In case the rebase brought in a commit that updated the submodules, we
should update them after the rebase instead of before. Otherwise the
submodules will be out of date compared to the source.
BUG=None
TEST=None
Change-Id: I398755abc76e9e6672c71ff1fe358a7c201c09fb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3403133
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
We currently have a lot of merges with just 1 or 2 commits in them.
That's a lot of noise.
This CL makes the builder wait until we got at least 5 commits to
merge. This will delay merge commits being created, but our merge
process is usually a few days behind due to CQ delays, so it does
not matter much.
The MIN_COMMIT_COUNT variable can be overwritten when triggering
a manual build in Fusion.
BUG=b:215389423
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: Ia24ee0b615ad402060775530e9eb0def7742200e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3402501
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Since we have a chain of commits, we need to set all of
them to CQ+1 for the dry run to happen.
Unfortunately, the CQ will only ever run a single build
for a chain of commits. So the dry run will only happen
when all builds have finished.
BUG=b:213932296
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I1883b22ab97da604a4188528ccfb3231a23cb7f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3384220
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
We can use the gerrit api to detect previous merge commits
the bot has made, then use those as the base for the next merge.
This will create smaller merges and is easier to manage as we do not
have to abandon so many changes.
One complication is detecting possible cherry picks the oncall may
have added to the previous merge commit.
Only open merge commits are taken into account.
BUG=b:213932296
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I795109be4c45bec49068c3e292a3947505755442
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3381832
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
This allows us to manually specify the merge target when triggering
the Kokoro job to do partial merges.
BUG=b:209034086
TEST=None
Change-Id: If68231b46981dec679445dde50e1fb5ff66e57c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3340211
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Change-Id: Ifb00dc70e8a2af690722a40cbdcd9836b950d929
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3339182
Reviewed-by: Anton Romanov <romanton@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
Auto-Submit: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This prevents gerrit from claiming we are forging email addresses.
BUG=b:209034086
TEST=None
Change-Id: I9f5bab0684f40ad5220d8c88d89a34e23facab99
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321734
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
There can be leftover files from a previous build.
BUG=b:209034086
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I0f7223bc6dd1769db04408f595764931c3ff2af8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321730
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
- Adds the needed job config
- Makes a copy of the script before git checkout
- Follow naming scheme of other build scripts
- Add reviewer when uploading
BUG=b:209034086
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I279ff6aca7c3c304354a238ebd5c40b4dd272385
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321726
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
The Kokoro build script sets up access to the chromium
gerrit, creates the merge commit and uploads it.
BUG=b:209034086
TEST=./ci/kokoro/simulate merge-to-chromeos.sh
Change-Id: Ia686b937c3d6b0b744d5c2162eb2847c5952e977
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3318793
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This vastly improves iterative build times and enables more flexible
usage of the container.
BUG=None
TEST=./tools/dev_container cargo build
First run will build everything. Second run will finish right away.
Change-Id: I9b4eeee0689f0e9d07f0a32f846d21ab42f689f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3292100
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
By default Kokoro will use changes as they come from gerrit, which may
have an outdated parent. We want to make sure that we are always testing
against tip of tree so we are confident the presubmit test result will
reflect the postsubmit results.
BUG=b:202275156
TEST=Tested in Kokoro
Change-Id: I2a1d2860a361eee741dc522c9859e32890ff15bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229947
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
One more fix. Everything seems to work, but the remote is rejecting
an update of the active main branch. Hopefully the last fix needed.
BUG=b:199692764
TEST=Testing in prod.
Change-Id: I4f341cc374080454317cb3ac29cade59fec6aeee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229958
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
We were in the wrong directory.
BUG=b:199692764
TEST=Testing in prod...
Change-Id: Idbc60f7f6084109220186e56cd7fad2e6553da9a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229950
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This is hard to test. I verified that kokoro can access
the github token and use it to access github by adding
a git clone into the x86 presubmit build file in patchset 3.
But we can only really test the full process by submitting this
CL and have it pass through the continuous builder.
BUG=b:199692764
TEST=Needs to be tested in prod...
Change-Id: Ifdcd385552c61ecea75a43fb71b08950e8b036ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229941
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This replaces the existing tooling at ./ci/ with the new tooling
at ./tools.
BUG=b:199951064
TEST=./ci/kokoro/simulate_all
Change-Id: Id0296d5a46ca0f3eb0f655d85d1aed801ee3b434
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3226275
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Somehow, after rsyncing the whole directory, git will no longer work in the
copy. So instead, we are now using 'git clone'.
This changes the behavior to not include local workspace modifications in
kokoro simulations. Which is ok, we have test_all for that and the behavior
will more closely match what's happening in kokoro presubmits.
BUG=None
TEST=./ci/simulate_all
Change-Id: I439b9eadcac65d99782e2b0eb869519abb2ada37
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3152425
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
We no longer need to build dependencies from source and provide them
via pkg-config, this simplifies much of our builder container setup.
As debian bullseye got promoted to stable, we can now also simplify
further by just using bullseye stable instead of mixing stable and
testing packages.
BUG=b:181359683
TEST=./test_all
Change-Id: I2ce61992d5cfe6eb5dc3f0ec61920dcc5455ca40
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3141772
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Instead of requiring crosvm to be checked out via the chromeos manifest
to access dependencies in the ChromeOS monorepo, this change adds
git submodules to third_party/.
The CI scripts and Cargo.toml are updated to use the new paths.
BUG=b:194336213
TEST=git clone --recursive https://chromium.googlesource.com/chromiumos/platform/crosvm
cd crosvm
cargo test
./test_all
./ci/kokoro/simulate_all
Change-Id: I9859d18176e21909ac3a140976fbd67cc14129bb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3049003
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
This will introduce a little instability into our test infrastructure
since tests will no longer be hermetic, but build against the latest
ToT version of our chromeos dependencies.
It will however, make our day-to-day workflows a lot easier. A lot of
changes are made across repository boundaries and require
annoying extra steps since Kokoro does not understand Cq-Depend.
BUG=b:185917395
TEST=./ci/kokoro/simulate_all passes
Change-Id: I3c3cae20d418574a3db53cc74d81245b87c228db
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2840057
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Update Kokoro manifest and Cargo.lock.
This change does not affect ebuild builds, so it can be safely chumped
to get Kokoro green.
BUG=None
TEST=./ci/kokoro/simulate_all passes
Change-Id: If4e2b849f12d4e223dcfe6dd5bb48e034067b466
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2840055
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Some judgement calls were made about unnecessary wrapping. Usually they
would get resolved by removing the wrapping or returning a convenient
error, but the ones that returned results for consistency with other
functions were added to the allow list.
The error handling in the usb code had a lot of unit error types which
is now a clippy lint. This was resolved by either removing the result
entirely or returning a convenient error.
The field_reassign_with_default lint is faulty and was added to the list
of supressions. This affected virtio-wayland code.
BUG=b:179277332
TEST=cargo clippy with rustc 1.50+
Change-Id: Ie812cdeaf7c42f4f2b47b1dc87f05a7c87a60f8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2757510
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Kokoro was failing since http://crrev.com/c/2744968 depends on recent
cras changes that were not included in the current Kokoro manifest.
BUG=None
TEST=./ci/kokoro/simulate_all passes
Change-Id: I470844af73b9f89ac8cbf9f5fcf8706d01924279
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2746014
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This change updates the test VM to debian bullseye, primarily to enable
io_uring features that are not available in the 4.16 kernel shipping
with debian buster.
We are not updating the builder containers to bullseye, as newer
versions of qemu fail to start on kokoro. See b/181359683
Since it was needed for debugging this CL, it includes a feature to
debug kokoro builders via SSH. It can be enabled by setting
DEBUG_SSH_KEY from the fusion2 UI.
BUG=b:178228512
TEST=./test_all passes
Change-Id: I22f1678577183a0aace80245a87c31907221cc09
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2721062
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This CL fixes some of the issues that prevented the
test system from running on other workstations.
- Builders run by Kokoro will not use a scratch mount.
It's not needed, and will cause issues if stored on
/tmp with noexec set.
- Running podman with label=disable to prevent selinux
issues.
The run_tests implementation has been moved to a separate
file and updated with parsing of cargo test output. This
allows simpler test output and integration with the
test results UI in sponge.
The sponge test UI can make it much easier to see which
tests failed, and to find the log of that test case.
This CL also includes an ./ci/kokoro/uprev script to
uprev the manifest versions. And runs the uprev.
BUG=b:174861002
TEST=Tested by forcing a kokoro build with this CL
Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Fixes some nits that cause kokoro builds to fail and updates
the Kokoro configs to match the new Kokoro-side configs
submitted in cl/355894530.
With this CL Kokoro v2 should be passing and can be enabled as
a presubmit.
BUG=b:179289824
TEST=Kokoro passes
Change-Id: I8c9259407bf1eb932b05376b35cb88d2e7d6058c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2678792
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Adds the crosvm-side infrastructure to build and test
in kokoro.
There is a build script for testing on x86, aarch64
and a separte script for analysis (clippy, fmt).
These will run in parallel on Kokoro. To test the
scripts locally, a simulate script is provided.
Runtime on my workstation:
- aarch64: 10m
- x86: 2:30m
- analysis: 1:40m
BUG=b:177951955
TEST=./ci/kokoro/simulate_all
Change-Id: I2f666ec768e6c3391a258dc7f0cbd999ad9b2fb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2654413
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
