mirrors/crosvm
1
0
Fork 0
mirror of https://chromium.googlesource.com/crosvm/crosvm synced 2024-10-24 21:23:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
crosvm/ci/Makefile

59 lines
1.8 KiB
Makefile
Raw Normal View History

native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
# Copyright 2021 The Chromium OS Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
# found in the LICENSE file.
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
#
# Builds docker images for the crosvm builders.
# Run the `upload` target to upload the images to the container registry
# (provided you are authorized to upload them).
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
#
# Images are always built with docker (since buildkit is a lot faster than
# podman/buildah). But we do automatically pull images into podman if podman
# is installed.
export DOCKER_BUILDKIT=1
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
TAG_BASE=gcr.io/crosvm-packages
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
TAG_VERSION=$(shell cat image_tag)
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
DOCKER ?= docker
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
all: crosvm_builder crosvm_aarch64_builder
upload: all
Add Kokoro for ci/builders Adds the crosvm-side infrastructure to build and test in kokoro. There is a build script for testing on x86, aarch64 and a separte script for analysis (clippy, fmt). These will run in parallel on Kokoro. To test the scripts locally, a simulate script is provided. Runtime on my workstation: - aarch64: 10m - x86: 2:30m - analysis: 1:40m BUG=b:177951955 TEST=./ci/kokoro/simulate_all Change-Id: I2f666ec768e6c3391a258dc7f0cbd999ad9b2fb1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2654413 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Stephen Barber <smbarber@chromium.org>
2021-01-27 00:14:41 +00:00
$(DOCKER) push $(TAG_BASE)/crosvm_base:$(TAG_VERSION)
$(DOCKER) push $(TAG_BASE)/crosvm_builder:$(TAG_VERSION)
$(DOCKER) push $(TAG_BASE)/crosvm_aarch64_builder:$(TAG_VERSION)
$(DOCKER) push $(TAG_BASE)/crosvm_test_vm_amd64:$(TAG_VERSION)
$(DOCKER) push $(TAG_BASE)/crosvm_test_vm_arm64:$(TAG_VERSION)
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
crosvm_base:
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
cd $@ && $(DOCKER) build -t $(TAG_BASE)/$@:$(TAG_VERSION) .
crosvm_builder: crosvm_base crosvm_test_vm_amd64
cd $@ && $(DOCKER) build \
-t $(TAG_BASE)/$@:$(TAG_VERSION) \
--build-arg TAG=$(TAG_VERSION) \
.
ifneq (, $(shell command -v podman))
podman pull docker-daemon:$(TAG_BASE)/$@:$(TAG_VERSION)
endif
crosvm_aarch64_builder: crosvm_base crosvm_test_vm_arm64
cd $@ && $(DOCKER) build \
-t $(TAG_BASE)/$@:$(TAG_VERSION) \
--build-arg TAG=$(TAG_VERSION) \
.
ifneq (, $(shell command -v podman))
podman pull docker-daemon:$(TAG_BASE)/$@:$(TAG_VERSION)
endif
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
crosvm_test_vm_amd64:
cd crosvm_test_vm && \
$(DOCKER) build -t $(TAG_BASE)/$@:$(TAG_VERSION) --build-arg VM_ARCH=amd64 .
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
crosvm_test_vm_arm64:
cd crosvm_test_vm && \
$(DOCKER) build -t $(TAG_BASE)/$@:$(TAG_VERSION) --build-arg VM_ARCH=arm64 .
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
.PHONY: all crosvm_base crosvm_builder crosvm_aarch64_builder
Reference in a new issue Copy permalink