mirrors/crosvm
1
0
Fork 0
mirror of https://chromium.googlesource.com/crosvm/crosvm synced 2025-02-11 12:35:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
crosvm/ci/README.md

37 lines
1.5 KiB
Markdown
Raw Normal View History

Kokoro: Extensive polishing and bugfixing This CL fixes some of the issues that prevented the test system from running on other workstations. - Builders run by Kokoro will not use a scratch mount. It's not needed, and will cause issues if stored on /tmp with noexec set. - Running podman with label=disable to prevent selinux issues. The run_tests implementation has been moved to a separate file and updated with parsing of cargo test output. This allows simpler test output and integration with the test results UI in sponge. The sponge test UI can make it much easier to see which tests failed, and to find the log of that test case. This CL also includes an ./ci/kokoro/uprev script to uprev the manifest versions. And runs the uprev. BUG=b:174861002 TEST=Tested by forcing a kokoro build with this CL Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712 Reviewed-by: Zach Reizner <zachr@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-02 23:18:10 +00:00
# CrosVM Continuous Integration
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
Kokoro: Extensive polishing and bugfixing This CL fixes some of the issues that prevented the test system from running on other workstations. - Builders run by Kokoro will not use a scratch mount. It's not needed, and will cause issues if stored on /tmp with noexec set. - Running podman with label=disable to prevent selinux issues. The run_tests implementation has been moved to a separate file and updated with parsing of cargo test output. This allows simpler test output and integration with the test results UI in sponge. The sponge test UI can make it much easier to see which tests failed, and to find the log of that test case. This CL also includes an ./ci/kokoro/uprev script to uprev the manifest versions. And runs the uprev. BUG=b:174861002 TEST=Tested by forcing a kokoro build with this CL Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712 Reviewed-by: Zach Reizner <zachr@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-02 23:18:10 +00:00
Crosvm has a complex set of dependencies and requirements on the host machine to
successfully build and run test cases. To allow for consistent testing in our
continuous integration system (kokoro) and reproduction of those tests locally,
we provide docker containers containing the build toolchain and a VM for
testing.
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
Kokoro: Extensive polishing and bugfixing This CL fixes some of the issues that prevented the test system from running on other workstations. - Builders run by Kokoro will not use a scratch mount. It's not needed, and will cause issues if stored on /tmp with noexec set. - Running podman with label=disable to prevent selinux issues. The run_tests implementation has been moved to a separate file and updated with parsing of cargo test output. This allows simpler test output and integration with the test results UI in sponge. The sponge test UI can make it much easier to see which tests failed, and to find the log of that test case. This CL also includes an ./ci/kokoro/uprev script to uprev the manifest versions. And runs the uprev. BUG=b:174861002 TEST=Tested by forcing a kokoro build with this CL Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712 Reviewed-by: Zach Reizner <zachr@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-02 23:18:10 +00:00
## Implementation Overview
Directories:
ci: Quote underscores. My editor doesn't like them, and probably they should be. BUG=None TEST=read it. Change-Id: I40041e41841e47cb386fcd12a4622efeca33474d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3215932 Reviewed-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Tested-by: Dennis Kempin <denniskempin@google.com>
2021-10-11 04:57:26 +00:00
- `ci/build_environment`: Contains tooling for building the dependencies of
kokoro: format README.md with Google style. BUG=None TEST=Read it. Change-Id: I477a4d18255c8cc5ed2e4eb903d6651615473ad6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2902914 Reviewed-by: Dennis Kempin <denniskempin@google.com> Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> Commit-Queue: Junichi Uekawa <uekawa@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com>
2021-05-18 08:16:36 +00:00
crosvm.
ci: Quote underscores. My editor doesn't like them, and probably they should be. BUG=None TEST=read it. Change-Id: I40041e41841e47cb386fcd12a4622efeca33474d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3215932 Reviewed-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Tested-by: Dennis Kempin <denniskempin@google.com>
2021-10-11 04:57:26 +00:00
- `ci/crosvm_aarch64_builder`: An x86 docker image to cross-compile for aarch64
kokoro: format README.md with Google style. BUG=None TEST=Read it. Change-Id: I477a4d18255c8cc5ed2e4eb903d6651615473ad6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2902914 Reviewed-by: Dennis Kempin <denniskempin@google.com> Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> Commit-Queue: Junichi Uekawa <uekawa@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com>
2021-05-18 08:16:36 +00:00
and test with user-space emulation.
ci: Quote underscores. My editor doesn't like them, and probably they should be. BUG=None TEST=read it. Change-Id: I40041e41841e47cb386fcd12a4622efeca33474d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3215932 Reviewed-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Tested-by: Dennis Kempin <denniskempin@google.com>
2021-10-11 04:57:26 +00:00
- `ci/crosvm_base`: Docker image shared by `crosvm_builder` and
`crosvm_aarch64_builder`
- `ci/crosvm_builder`: A native docker image for building and testing crosvm
- `ci/crosvm_test_vm`: Dockerfile to build the VM included in the builder
kokoro: format README.md with Google style. BUG=None TEST=Read it. Change-Id: I477a4d18255c8cc5ed2e4eb903d6651615473ad6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2902914 Reviewed-by: Dennis Kempin <denniskempin@google.com> Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> Commit-Queue: Junichi Uekawa <uekawa@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com>
2021-05-18 08:16:36 +00:00
containers.
ci: Quote underscores. My editor doesn't like them, and probably they should be. BUG=None TEST=read it. Change-Id: I40041e41841e47cb386fcd12a4622efeca33474d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3215932 Reviewed-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Tested-by: Dennis Kempin <denniskempin@google.com>
2021-10-11 04:57:26 +00:00
- `ci/kokoro`: Configuration files and build scripts used by Kokoro to run
kokoro: format README.md with Google style. BUG=None TEST=Read it. Change-Id: I477a4d18255c8cc5ed2e4eb903d6651615473ad6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2902914 Reviewed-by: Dennis Kempin <denniskempin@google.com> Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> Commit-Queue: Junichi Uekawa <uekawa@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com>
2021-05-18 08:16:36 +00:00
crosvm tests.
Kokoro: Extensive polishing and bugfixing This CL fixes some of the issues that prevented the test system from running on other workstations. - Builders run by Kokoro will not use a scratch mount. It's not needed, and will cause issues if stored on /tmp with noexec set. - Running podman with label=disable to prevent selinux issues. The run_tests implementation has been moved to a separate file and updated with parsing of cargo test output. This allows simpler test output and integration with the test results UI in sponge. The sponge test UI can make it much easier to see which tests failed, and to find the log of that test case. This CL also includes an ./ci/kokoro/uprev script to uprev the manifest versions. And runs the uprev. BUG=b:174861002 TEST=Tested by forcing a kokoro build with this CL Change-Id: I0cba9bb68915e2558a4ea6061dd9ba0a7050421b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2669712 Reviewed-by: Zach Reizner <zachr@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-02 23:18:10 +00:00
Scripts:
ci: Quote underscores. My editor doesn't like them, and probably they should be. BUG=None TEST=read it. Change-Id: I40041e41841e47cb386fcd12a4622efeca33474d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3215932 Reviewed-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Tested-by: Dennis Kempin <denniskempin@google.com>
2021-10-11 04:57:26 +00:00
- `ci/aarch64_builder`: Script to start the `crosvm_aarch64_builder` container
- `ci/builder`: Script to start the `crosvm_builder` container
- `ci/run_container.sh`: Implementation behind the above scripts.
- `test_runner.py`: Implementation behind the `./test_all` script.
native and aarch64 cross-compile containers This CL adds the foundation for running tests consistently in Kokoro and locally, for both x86 and aarch64. The crosvm_builder is similar to the original image from docker/crosvm.Dockerfile. The main difference is that ChromeOS dependencies are not compiled into the container, but built at runtime. The crosvm_aarch64_builder installs the build enviornment to cross-compile crosvm for aarch64. The tests are run with user-space emulation using qemu-aarch64-static. See ci/README.md for instructions on how to use these builders. Tests on aarch64 cannot all be run using user-space emulation. We will need a VM to pass all smoke tests, this work is tracked in b/177228167. BUG=b:177133814 TEST=Tested by running ./ci/builder bin/smoke_test ./ci/builder cargo test ./ci/aarch64_builder cargo build ./ci/aarch64_builder cargo test -p tempfile Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994 Reviewed-by: Zach Reizner <zachr@chromium.org> Reviewed-by: Stephen Barber <smbarber@chromium.org> Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com>
2020-12-03 18:00:47 +00:00
### Building and uploading a new version of builders
The docker images for all builders can be built with `make` and uploaded with
`make upload`. Of course you need to have docker push permissions for
Add VM to builders The VM allows both the x86 and aarch64 builders to run tests which require access to kernel devices that docker would prevent us from accessing. The VM is automatically started in the endpoint script, and the environment is set up to execute `cargo test` binaries inside the VM. This allows for convenient interactive usage. The VM is also integrated into the run_tests scripts, executing tests that do not require special privileges first, while the VM is still booting up, then execuing the remaining tests in the VM. BUG=b:177228167,b:177951276 TEST=These test calls pass: ./ci/builder --vm ./run_tests ./ci/aarch64_builder --vm ./run_tests ./run_tests Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766 Tested-by: Dennis Kempin <denniskempin@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-01-21 19:06:44 +00:00
`gcr.io/crosvm-packages/` for the upload to work.
Reference in a new issue Copy permalink