Using HEAD for both calls will cause update-dry-runs to use the
merge branch as a target to do the merge, creating the dry run on top
of the merge.
BUG=None
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-merge-into-chromeos.sh
Change-Id: Ib466272d3acce30888e1e3c5db853be9ded46106
Change-Id: Ifc40ffb81a61a75326b296419a4337c49a8fd7db
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3688803
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
We will be merging from the upstream repo into the cros repo. The
script can be run from both an upstream and a cros checkout, it
will setup the required remotes.
The CLI changes a little. We now require a revision to merge from
needs to be specified, since the script is not setting up the
upstream remote.
CI jobs will simply use HEAD, as the version to merge is already
checked out by the CI system.
BUG=b:233913643
TEST=
./merge_bot update-merges HEAD
./merge_bot update-merges origin/main
./merge_bot update-dry-runs HEAD
Change-Id: I247e1edc3ad951becf5d59b63efd74914bb2205f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3687222
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Some recent changes introduced a bug that breaks reusage of
containers. Instead of trying to restart a stopped container,
we delete it instead and start a fresh one.
The code was refactored to allow for testing. The tests run
against the real docker service and are run as part of the health
checks in kokoro / luci.
This CL also gets rid of the custon command line parsing since
luci can now run it with argh available.
BUG=b:234402839
TEST=./tools/dev_containes --self-test [--podman]
Change-Id: Id5eaf0ea83ff07433f8f57cb652b5c393dcb9da3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3681399
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Add the --no-progress option so that choco does not print ten thousand
lines of "Progress: ... X%" messages in the CI logs.
BUG=None
TEST=kokoro CI
Change-Id: I7ec414f4baeeb530f0dc103b256807c7f642ed53
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657051
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Previously, the script would install the python package without a
version specifier, then attempt to install a specific version (3.9.0),
but this would fail because the first step installed a newer Python
version:
A newer version of python (v3.10.4) is already installed.
Use --allow-downgrade or --force to attempt to install older versions,
or use side by side to allow multiple versions.
Chocolatey installed 0/1 packages. 1 packages failed.
Remove the first unversioned step so we always get the requested version
of Python only.
BUG=None
TEST=kokoro CI
Change-Id: I95b224926ee9ebcaacdb7d6c2716381a46ec467c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3657850
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Kokoro will treat the files as build artifacts and tries to rsync
the ~80GB to the executor. See b/30309837.
BUG=b:2089374426
TEST=None
Change-Id: I71df011025a7cff664a134bdfbf5783709365358
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3611872
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Re-use the existing logic to install python, setup the source and
most importantly, enable SSH debugging.
BUG=b:226975041
TEST=None
Change-Id: I0f8b0251bea163145ba1611b3b36c4bb7c8ee968
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606979
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
The ChromeOS builds take a lot longer than other crosvm builds.
BUG=b:226975041
TEST=None
Change-Id: I2cf9071aebba10e46a7b63fed0cafbec7a8b06b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3606970
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This adds an experimental build script that will build crosvm for
ChromeOS. This will allow us to catch issues on the ChromeOS build of
crosvm earlier in the process.
BUG=b:226975041
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-chromeos.sh
Change-Id: I4a1d906a44191e78f4e656fe76c989bd7e1b6977
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3600764
Reviewed-by: Dmitry Torokhov <dtor@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This change adds python type and formatting checks and
consolidates code health checks in ./tools/health-check.
Dealing with relative imports in python is tricky, so
we are making ./tools/impl a proper package with no
directly executable files.
Some of the bash shorthands in ./tools had to be converted
to python for this.
To make the new checks pass, we run the formatter and fix
some mypy type checks.
TEST=./tools/health-check
BUG=b:218559722,b:219965702
Change-Id: Ie18d3d6dd2f5a033141e167a6e1aa762791941d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3558592
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
If the version is installed, but not the default selected, pyenv
can fail.
BUG=None
TEST=Testing in prod..
Change-Id: I5063051dd300f9dbbb057e429a84062341b3324c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585859
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
The script broke after the formatter wrapped the string.
BUG=None
TEST=./ci/kokoro/simulate.py build-merge-into-chromeos.sh
Change-Id: Id6e972d786d5a58f6cbb43f80d02f0aa26483967
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3585858
Reviewed-by: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
The new merge bot is written in python to allow us to improve the
merge process.
The new process is as follows:
- The merge bot will only create a new merge when the previous
ones have been submitted.
- When the bot creates a new merge, it'll split the merge commits
into smaller changes to make bisection easier.
This allows us to only ever deal with one set of merge commits
instead of piling on new merges day after day.
For dry runs, the new process is:
- The bot will check existing dry runs and abandon them if the CQ
is done.
- The bot will vote Verified+-1 to indicate if the dry run passed.
- If no more dry runs are active, it'll create a new one.
This ensures we always have a dry run in the CQ and can easily see
in gerrit which ones have passed.
The python script will also allow us to add convenience features
for oncall for retrying bratches, etc.
BUG=b:227347397
TEST=The script can be run locally:
./tools/chromeos/merge_bot update-merges
./tools/chromeos/merge_bot update-dry-runs
Change-Id: I51fc0741f5032868ad07f5e926e570e29cc461fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3583254
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
The recent cherry-pick of a revert has diverged the tree and caused
the commits inside the merge to be rebased as well, which uploads new
gerrit CLs for each commit.
BUG=None
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-x86_64.sh
Change-Id: I01af93c989ab0aba4c2d599d0625f921315a0600
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3566120
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Also fixes few warning introduced during upstream which linux clippy
run does not catch.
test: py tools\clippy
bug: 226966790
Change-Id: I6979f87be5d6d34727afabafcaccb57631975567
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3553401
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
This allows the bot to account for manually cherry-picked commits.
To make rebases work properly, we had to fix the commit message
trailers to be added in a separate paragraph. This ensures
git inspect-trailers can add and find the Commit-Id trailer.
BUG=b:226975273
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-merge-into-chromeos.sh
Change-Id: If008be1f1c671ef1d79c6f223f10ea0e8f6742b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3561015
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Maciek Swiech <drmasquatch@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Does couple of things
- move tube under right directory
- cleanup cros_async dependency
- export tube module from base
- enable base windows tests
- add verbose flag to test runner
test: built on window and presubmit
bug: b:213151429
bug: b:213153157
Change-Id: I0d1906d7edcc3352db53325ea7527550eaf00895
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3550296
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Change-Ids can refer to multiple CLs when they are cherry-picked.
Use the 'id' field instead wich contains repo and branch info and
is unique across gerrit.
BUG=None
TEST=./ci/kokoro/simulate.py ./ci/kokoro/build-merge-into-chromeos.sh
Change-Id: I7074e310ec7f795527610010c849acceaeb0eb06
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3556231
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Check whether or not the rebase succeeds and fix a check that the script
is being run by kokoro.
BUG=None
TEST=tools/presubmit
Change-Id: I59ae6604c3535238f40a7df20c43b9db9e357fe3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3546578
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: David Stevens <stevensd@chromium.org>
Run tests for sys_util_core, poll_token_derive and balloon_control on
windows.
Using dotfiles to disable/serialize test runs of a subset of crates does
not work well with third party crates as it forces us to commit the dot
file to the crate.
The patch modifies and uses the script that runs linux tests.
This patch also allows us to
- build/test child crate even if parent crate has disabled build/test.
- avoid building crosvm if it is not explicitly specified.
RIP short lived .windows_build_test_skip. You allowed us to run noop
kokoro tests.
Test: py .\tools\impl\test_runner.py --arch x86_64
Bug: b:215610772
Change-Id: Icc6d04ffd7c0c33d4f60aeac16fc7d23881c387d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3459809
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Use the new common.py utilities to make turn this bash script into
python. The behavior should be the same.
BUG=b:218559770
TEST=./tools/presubmit
Change-Id: Id1412c2076089fd21f280959d6cde1f4cb64e163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3469049
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
To help find missing feature cfg checks, add a build with all features
disabled to the CI scripts.
BUG=None
TEST=tools/presubmit
Change-Id: I98f38ecc3bf90cbeb081a1fafe4f2f540e4fc348
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3437362
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Build config sets timeout to 40 minutes.
helper script relies on choco to install rustup, which is used
to install rust toolchain.
Bug: 213603713
Test: Tested the toolchain setup part of the script in a
codelab repo.
Change-Id: Ibf139bbc4ff6fe6c811652767dd885584414dda6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3421515
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Brandon Amiosci <brandonamiosci@google.com>
Commit-Queue: Vikram Auradkar <auradkar@google.com>
Does the same thing, but in python. This should allow us to make
this work on windows as well.
BUG=b:216833094
TEST=./simulate.py build-x86_64.sh
Change-Id: If6eb214c2a3185fbb60a636b89cd03ecf28b42e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3425651
Reviewed-by: Vikram Auradkar <auradkar@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
The dry run merge is re-created every day and automatically
set to CQ+1. This allows us to check for bad crosvm changes
while we submit parts of the merge chain.
BUG=b:215741244
TEST=./tools/ci/simulate merge-into-chromeos.sh
Change-Id: I0773a99f9858962e9f25bebff1b233d455d33cdd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3417508
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
In case the rebase brought in a commit that updated the submodules, we
should update them after the rebase instead of before. Otherwise the
submodules will be out of date compared to the source.
BUG=None
TEST=None
Change-Id: I398755abc76e9e6672c71ff1fe358a7c201c09fb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3403133
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
We currently have a lot of merges with just 1 or 2 commits in them.
That's a lot of noise.
This CL makes the builder wait until we got at least 5 commits to
merge. This will delay merge commits being created, but our merge
process is usually a few days behind due to CQ delays, so it does
not matter much.
The MIN_COMMIT_COUNT variable can be overwritten when triggering
a manual build in Fusion.
BUG=b:215389423
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: Ia24ee0b615ad402060775530e9eb0def7742200e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3402501
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Since we have a chain of commits, we need to set all of
them to CQ+1 for the dry run to happen.
Unfortunately, the CQ will only ever run a single build
for a chain of commits. So the dry run will only happen
when all builds have finished.
BUG=b:213932296
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I1883b22ab97da604a4188528ccfb3231a23cb7f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3384220
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
We can use the gerrit api to detect previous merge commits
the bot has made, then use those as the base for the next merge.
This will create smaller merges and is easier to manage as we do not
have to abandon so many changes.
One complication is detecting possible cherry picks the oncall may
have added to the previous merge commit.
Only open merge commits are taken into account.
BUG=b:213932296
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I795109be4c45bec49068c3e292a3947505755442
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3381832
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
This allows us to manually specify the merge target when triggering
the Kokoro job to do partial merges.
BUG=b:209034086
TEST=None
Change-Id: If68231b46981dec679445dde50e1fb5ff66e57c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3340211
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Change-Id: Ifb00dc70e8a2af690722a40cbdcd9836b950d929
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3339182
Reviewed-by: Anton Romanov <romanton@google.com>
Commit-Queue: Anton Romanov <romanton@google.com>
Auto-Submit: Anton Romanov <romanton@google.com>
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
This prevents gerrit from claiming we are forging email addresses.
BUG=b:209034086
TEST=None
Change-Id: I9f5bab0684f40ad5220d8c88d89a34e23facab99
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321734
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
There can be leftover files from a previous build.
BUG=b:209034086
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I0f7223bc6dd1769db04408f595764931c3ff2af8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321730
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
- Adds the needed job config
- Makes a copy of the script before git checkout
- Follow naming scheme of other build scripts
- Add reviewer when uploading
BUG=b:209034086
TEST=./ci/kokoro/simulate build-merge-into-chromeos.sh
Change-Id: I279ff6aca7c3c304354a238ebd5c40b4dd272385
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3321726
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
The Kokoro build script sets up access to the chromium
gerrit, creates the merge commit and uploads it.
BUG=b:209034086
TEST=./ci/kokoro/simulate merge-to-chromeos.sh
Change-Id: Ia686b937c3d6b0b744d5c2162eb2847c5952e977
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3318793
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
This vastly improves iterative build times and enables more flexible
usage of the container.
BUG=None
TEST=./tools/dev_container cargo build
First run will build everything. Second run will finish right away.
Change-Id: I9b4eeee0689f0e9d07f0a32f846d21ab42f689f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3292100
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Junichi Uekawa <uekawa@chromium.org>
By default Kokoro will use changes as they come from gerrit, which may
have an outdated parent. We want to make sure that we are always testing
against tip of tree so we are confident the presubmit test result will
reflect the postsubmit results.
BUG=b:202275156
TEST=Tested in Kokoro
Change-Id: I2a1d2860a361eee741dc522c9859e32890ff15bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229947
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
One more fix. Everything seems to work, but the remote is rejecting
an update of the active main branch. Hopefully the last fix needed.
BUG=b:199692764
TEST=Testing in prod.
Change-Id: I4f341cc374080454317cb3ac29cade59fec6aeee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3229958
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
