Commit graph

1944 commits

Author SHA1 Message Date
Dennis Kempin
aa0717443e Add VM to builders
The VM allows both the x86 and aarch64 builders to run tests which
require access to kernel devices that docker would prevent us
from accessing.
The VM is automatically started in the endpoint script, and the
environment is set up to execute `cargo test` binaries inside the VM.

This allows for convenient interactive usage.
The VM is also integrated into the run_tests scripts, executing tests
that do not require special privileges first, while the VM is still
booting up, then execuing the remaining tests in the VM.

BUG=b:177228167,b:177951276
TEST=These test calls pass:
./ci/builder --vm ./run_tests
./ci/aarch64_builder --vm ./run_tests
./run_tests

Change-Id: I20bf2cd848e944d411b97f1f8356279e312d8583
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2642766
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-02-05 03:01:53 +00:00
Daniel Verkamp
9cdd5fb6f0 devices: fix build with audio feature disabled
When building without the "audio" feature, an import of virtio::snd in
pci_device failed to compile; wrap it in a feature check to fix this.
This fixes the crosvm fuzzer build.

BUG=chromium:1174254
TEST=cargo build --no-default-features
TEST=build crosvm with amd64-generic asan profile

Change-Id: Iecd549c313d6210ee90e9e405c02a9e7581e1141
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2674260
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
2021-02-04 17:44:51 +00:00
Dennis Kempin
5545c907d6 Add COIL presubmit to crosvm
Details:
https://docs.google.com/presentation/d/13mw1ECs65JSCh6EoDZeG5AF6hHGW0ZT4qPghPcCXIK8/edit#slide=id.p

I used the search_blocked_words.sh script to find words we are not
using and commented them out.

BUG=b:178821708
TEST=presubmit passes

Change-Id: Iac326f200a2a1d4a32df4f790872f37d8571fd42
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2673278
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-02-04 02:47:03 +00:00
Gurchetan Singh
293913c01a rutabaga_gfx: rutabaga_gralloc: a shimmering beacon of hope
rutabaga_gralloc is a cross-platform, Rust-based buffer
manager.

The rationale for this change is:

1) For the {cross-domain, wayland} context type, we need to
have a good story for the crucial "wl-dmabuf" feature.  As
minigbm has been thoroughly tested on ChromeOS and currently
powers the "wl-dmabuf" feature, it only makes sense for us to
have a path to minigbm for the cross-domain prototype.  This
will be used by Sommelier.

2) While minigbm allocation works well on Chromebooks, it is
not sufficient for cross-platform purposes.  For their Virtual
Graphics Interface (VGI) initiative, Android graphics
virtualization experts have expressed their desire for a Vulkan
based allocator.  This will to go alongside cros_gralloc in
minigbm, which is considered by many to be the ""world's
premiere gralloc implementation".

3) Android graphics virtualization experts have expressed their
desire for vkMapMemory(..) to be used when crosvm is in
multi-process mode.  Currently, only dma-buf mmap() is supported
for zero-copy blobs in multi-process mode.  dma-buf mmap() is not
guaranteed to work on Nvidia (a "must have" for Cuttlefish) or
any other driver for that matter (we *make* it work for ChromeOS).
Possibly only solution: vkMapMemory ;-)

With these goals in mind, here's a summary of the revelant changes:

* Renamed the {gpu_allocator.rs, GpuMemoryAllocator trait} to be
  {gralloc.rs, Gralloc trait}.

* Moved all GPU allocation out of the resources crate and into
  the rutabaga_gfx crate.  This will allow the resources crate to
  be focused on managing resources for virtual machines.

* Moved the gpu_buffer crate into the gralloc module in the
  rutabaga_gfx crate.  The same functionality is now under
  "minigbm.rs", "minigbm_bindings.rs" and "rendernode.rs"

* Added an optional dependency on vulkano.rs.  vulkano.rs is a safe
  Rust wrapper around the Vulkan api [a].  It's emphasis on type
  safety makes a good fit for crosvm, though there are other high
  quality crates out there (gfx-rs, ash.rs).  Though development
  has slowed down, it should satisfy goals (2) and (3) quite easily.

* Added a system_gralloc implementation based on memfd.  This can be
  used when minigbm or Vulkano features are not used, to replicate the
  highly useful "wl-shm" feature in Sommelier.  Astute observers will
  note this can also enable seamless Wayland windowing without GPU
  features for Android too.  Some minor changes to the base crate were
  needed.

* Cut down on the amount of DrmFormats to the subset needed by
  Sommelier and cros_gralloc.

* Moved checked arithmetic into it's own file.

* Internally renamed to "wl-dmabuf" feature to be the "minigbm"
  feature.  This is because "wl-dmabuf" has a dependency on minigbm.

* Small rutabaga_gfx cleanups

[a] https://github.com/vulkano-rs/vulkano/blob/master/DESIGN.md

BUG=b:146066070, b:173630595, b:150239451
TEST=launch virtual machine with 2D mode
TEST=launch virtual machine with 3D mode
TEST=run sommelier with "wl-dmabuf" and "wl-shm"

Change-Id: I693a39cef64cd98e56d843d3c60caa7983d4d6e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2626487
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2021-02-04 02:35:52 +00:00
Jorge E. Moreira
359e7de9a1 Add the VioS audio backend
It only supports playback streams, with capture streams to be added in
a different change.

BUG=b/171602855

Change-Id: Id9a5a560506f8fd026ef3ed83f8d14b29389e329
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2574813
Tested-by: Jorge Moreira Broche <jemoreira@google.com>
Commit-Queue: Jorge Moreira Broche <jemoreira@google.com>
Auto-Submit: Jorge Moreira Broche <jemoreira@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-02-01 23:47:02 +00:00
Daniel Verkamp
52e51b6539 docker: update platform2 commit and libchromeos deps
BUG=None
TEST=docker/build_crosvm_base.sh && docker/wrapped_smoke_test.sh

Change-Id: I870792ba340ff76d565adb45218c7af7a6a08d71
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2653869
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-30 01:21:46 +00:00
Chuanxiao Dong
f3050db7bc i8042: use address instead of offset for read and write
Offset right now means the offset from base address that the
device was accessed at. And address is the absolute address
of the device's access in its address space. The port 0x61 and
0x64 is the absolute port number.

BUG=None
TEST=boot VM and run reboot command

Change-Id: I51ee90e51b559254cf76f2bec4d174294961fe1c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2648858
Reviewed-by: Colin Downs-Razouk <colindr@google.com>
Reviewed-by: Alistair Delva <adelva@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Alistair Delva <adelva@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
2021-01-29 18:04:35 +00:00
Daniel Verkamp
66f975eec7 devices: virtio: ignore driver interrupt suppression
The arcvm guest has been observed to fail early in boot with a low
probability; the root cause seems to be a missing interrupt from the
virtio-block device causing the guest to wait forever for a notification
that does not happen.

In testing, it seems that ignoring the VIRTQ_AVAIL_F_NO_INTERRUPT flag
set by the guest avoids the issue; possibly there is a race in the
publishing of used descriptors versus the checking of this flag, but I
was not able to find a definitive solution.

Ignoring the flag means that we may occasionally trigger an interrupt
when the guest has indicated that one is not necessary; this is, at
worst, a slight performance degradation, not a correctness issue, since
the guest kernel will ignore spurious interrupts.

BUG=b:172975852
TEST=Restart arcvm in a loop without failures 1000+ iterations

Change-Id: I4c91ccc5dadc03dde43008ff20fae1c4592e19da
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2648009
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-01-29 02:43:40 +00:00
Tristan Muntsinger
486cffc41d Add and enable virtio multi-touch touchscreen device
BUG=b:124121375
TEST=compile and run

Change-Id: I795ec238cb4ba7551a98fdfd4258fcae38e8a7a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2439297
Tested-by: Tristan Muntsinger <muntsinger@google.com>
Commit-Queue: Tristan Muntsinger <muntsinger@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Auto-Submit: Tristan Muntsinger <muntsinger@google.com>
2021-01-29 01:58:47 +00:00
Ryo Hashimoto
18d5733e15 Revert "Relocate FDT to beginning of RAM"
This reverts commit 890ae4cdb8.

Reason for revert: breaks arc.Boot.vm on kukui-arc-r

Original change's description:
> Relocate FDT to beginning of RAM
>
> This change serves two purposes. The FDT needs to be in a location
> accessible by 32 bit devices. Secondly the FDT's location was dependent
> on the amount of RAM allocated to the guest prior to this change which
> made passing it's location to the BIOS challenging.
>
> By moving the FDT to a fixed location within the 32 bit accesible memory
> boundary, these problems have been fixed.
>
> BUG=b:177926450
> TEST=local boot of android virtual device on ARM host
> Change-Id: I89e481f66ddc8ffcf0e86f00fbfa92d407c48736
> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2641126
> Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
> Tested-by: Ram Muthiah <rammuthiah@google.com>
> Commit-Queue: Ram Muthiah <rammuthiah@google.com>

Bug: b:177926450
Bug: b:178427080
Change-Id: Iba629542d024fff84721ff434655c0c322020db3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2656215
Reviewed-by: Ryo Hashimoto <hashimoto@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Ryo Hashimoto <hashimoto@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Ryo Hashimoto <hashimoto@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-01-29 00:56:47 +00:00
Jorge E. Moreira
a91761a8a2 Add some constants and structs for the virtio-snd device
These will eventually be used when the virtio-snd device is
implemented as well as by the VioS audio backend.

BUG=b/174713663

Change-Id: Id9fe1527a128df0059805076c01e20ddb1ad2b72
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2574812
Auto-Submit: Jorge Moreira Broche <jemoreira@google.com>
Tested-by: Jorge Moreira Broche <jemoreira@google.com>
Commit-Queue: Jorge Moreira Broche <jemoreira@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-01-27 02:40:01 +00:00
Chirantan Ekbote
4a86ce877a cros_async: Simplify uring executor drop impl
Closing the uring fd should release the resources borrowed by the kernel
so there's no reason to call io_uring_enter to drive any pending IO to
completion.  Doing so may end up blocking the thread indefinitely.

Instead just make sure that the URingContext is dropped before anything
else.  Rust's drop order guarantees that struct fields are dropped in
the order of declaration so this just means putting it first.

Also, since the RawExecutor is wrapped in an Arc it may end up being
dropped from a different thread than the one that called run() or
run_until().  We know there are no other references so just clear the
cached thread_id so that we don't panic in the drop impl.  We don't
currently have any users that want to run the executor concurrently on
multiple threads so we'll just punt that problem until we actually need
to deal with it.

BUG=none
TEST=unit tests

Change-Id: Icf6a23fc433128dd00adbd56a715dbae24cd8ea2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643845
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-26 09:15:13 +00:00
Chirantan Ekbote
d1e8f146b1 cros_async: Replace notify_task with IORING_OP_NOP
IORING_OP_NOP is a no-op that does no IO and can be used to measure the
performance of the io_uring subsystem or to wake up a thread currently
blocked inside an io_uring_enter syscall. Use it instead of keeping a
separate eventfd.

BUG=none
TEST=unit tests

Change-Id: I3b3e93c653e1db747c90456c1d93abb979c25d34
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643844
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
2021-01-26 09:15:12 +00:00
Chirantan Ekbote
1fcc8a8f3c io_uring: Make UringContext Sync
Rather than requiring callers to wrap the entire UringContext in a big
lock, make UringContext Sync by internally using fine-grained locks:

* The submit queue is wrapped with a Mutex so that threads may still add
  operations to the submit queue even when a thread is blocked
  inside an io_uring_enter call.
* The completion queue internally uses a Mutex around the pending op
  data and completed count so that two threads don't end up trying to
  remove the same operation from the completed queue.

With this we can enable the await_uring_from_poll test.  This test
uncovered missing wakeups in the case where a uring operation is added
from one thread while the main uring thread is blocked inside an
io_uring_enter syscall. To deal with this, we call submit() whenever the
pending operation is polled and not yet submitted.

BUG=none
TEST=unit tests

Change-Id: I4c7dec65c03b7b10014f4a84d2cd16fe8758ea72
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643842
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-01-26 09:15:11 +00:00
Chirantan Ekbote
1e9974ff3e cros_async: Add the WeakWake trait
This is like future::task::ArcWake but uses Weak<T> instead of Arc<T>.
This prevents a circular reference where the RunnableQueue would hold an
active reference to the executor (via its waker) while the executor
owned the RunnableQueue, leading to memory leaks.

With this the executor should only be using weak references internally
and the only strong references should be in the public *Executor type.

Also fix the tests that were broken by this change.

BUG=none
TEST=unit tests

Change-Id: I3e9c007d533e154b2ad9d1e4d56a692f65da6aa0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643841
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: Chirantan Ekbote <chirantan@chromium.org>
2021-01-25 08:52:27 +00:00
Chirantan Ekbote
f1ce859903 cros_async: Refactor executors
There are a lot of changes in this one but these are the high-level
points:

* Both executors now support non-fd futures and it's now possible to
  start a poll operation on one thread and then await the result inside
  a UringExecutor on another thread.  The reverse doesn't work yet but
  will once we make UringContext sync.
* A few layers of indirection have been removed so hopefully both the
  implementation and the interface are simpler.
* The thread local magic is gone in favor of directly calling methods on
  an executor.  Executor handles can be cheaply cloned to make this
  easier.
* Heap allocations are limited to the spawn and spawn_local methods so
  it's possible to completely avoid heap allocations if callers only use
  the run_until method.
* The IoSource, Executor, FutureList traits are gone.

BUG=none
TEST=unit tests

Cq-Depend: chromium:2629068
Change-Id: I86053007929c36da66f3b2499cdefce0b5e9a180
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571401
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-01-25 08:52:27 +00:00
Ram Muthiah
890ae4cdb8 Relocate FDT to beginning of RAM
This change serves two purposes. The FDT needs to be in a location
accessible by 32 bit devices. Secondly the FDT's location was dependent
on the amount of RAM allocated to the guest prior to this change which
made passing it's location to the BIOS challenging.

By moving the FDT to a fixed location within the 32 bit accesible memory
boundary, these problems have been fixed.

BUG=b:177926450
TEST=local boot of android virtual device on ARM host
Change-Id: I89e481f66ddc8ffcf0e86f00fbfa92d407c48736
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2641126
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Ram Muthiah <rammuthiah@google.com>
Commit-Queue: Ram Muthiah <rammuthiah@google.com>
2021-01-23 02:22:28 +00:00
Dennis Kempin
b40ec47309 Make Kokoro happy
A few things happened:

- A recent minigbm change broke compilation for debian, so
  I am pinning the commit SHA for the Dockerfile.
- Newer rust versions come with new clippy errors, which I will
  have to look at in a separate change. For now I am pinning our
  rust version.
- Some repos have transitioned to a main branch. Fixed our uprev
  script to handle those.
- Renamed SYSROOT so we won't confuse pkg-config.

Now Kokoro is finally happy again!

BUG=None
TEST=wrapped_smoke_tests

Change-Id: I76294abe35fd84b305124158b3942a321651982d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643779
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-01-22 22:32:37 +00:00
Chia-I Wu
d562b1a788 gpu: bind mount /usr/share/vulkan for vulkan loader
The loader scans /usr/share/vulkan/icd.d for drivers.

BUG=b:176158397
TEST=vkcube

Change-Id: I63607148a534980479d1fd3aed8ea7b790c46c3a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2634994
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Chia-I Wu <olv@google.com>
Commit-Queue: Chia-I Wu <olv@google.com>
2021-01-22 13:28:34 +00:00
Chia-I Wu
694c53031a seccomp: gpu: allow vulkan loader and intel anv
BUG=b:176158397
TEST=vkcube

Change-Id: I11b5a97929fbb2c0328aca7d54b5882823b67209
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2634993
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Chia-I Wu <olv@google.com>
Commit-Queue: Chia-I Wu <olv@google.com>
2021-01-22 13:28:33 +00:00
Chia-I Wu
186908d641 virgl: use virgl_renderer_context_create_with_flags
When context_init is non-zero, pass it to virglrenderer by calling
virgl_renderer_context_create_with_flags.

BUG=b:176158397
TEST=vkcube

Change-Id: If9b8c60be7e6f1ef643077bd6a94aed0235a30f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2634992
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Chia-I Wu <olv@google.com>
Commit-Queue: Chia-I Wu <olv@google.com>
2021-01-22 13:28:32 +00:00
Dennis Kempin
2118d6faa3 Test Runner for crosvm
The ./run_tests script will select which tests to run on the host.
Unfortunately a lot of our tests require dependencies or access
to certain kernel devices. We cannot run all tests on all hosts.

The test runner works with the CI builders:

To run all x86 tests:
./ci/builder ./run_tests --all

To run tests on aarch64:
./ci/aarch64_builder ./run_tests --all

Or to just run tests that can be run on a standard debian buster
system without obscure chromeos dependencies:

./run_tests

Eventually, the test runner should run tests via the VM of the
aarch64_builder, currently only those tests that can be run with
user-space emulation are executed.

Currently some tests remain disabled in the runner, as they do not
pass in the CI builders yet. These will be enabled as the builders
are set up to allow for them to be run.

BUG=b:173833661
TEST=Ran the above commands.

Change-Id: Id69027793348f4d3f20adf5333d8bdfff1aa9c8b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633889
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-22 11:05:57 +00:00
Michael Hoyle
bc94c97300 vm_memory: Update memfd variable name
Update memfd -> shm to better match downstream.

BUG=b:176188689
TEST=./build_test

Change-Id: I32ba56fe89938ca58130ba02878bad2cb1f5012e
Merged-In: I7e8c91fec76babd2f6f10f9f739e1d7a181b65e7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633888
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2021-01-22 05:02:00 +00:00
Daniel Verkamp
c7236f2cf1 seccomp: allow USBDEVFS_CLEAR_HALT
This USB ioctl is used in usb_util::Device::clear_halt(), but it was not
allowed in the seccomp policy.

BUG=chromium:1167286
TEST=Attach Keyspan USA-19H USB serial adapter to Crostini

Change-Id: I625cde121a0a248046e476eecd732a98530811dc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633824
Reviewed-by: Matthew Blecker <matthewb@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Matthew Blecker <matthewb@chromium.org>
Commit-Queue: Matthew Blecker <matthewb@chromium.org>
2021-01-22 05:00:05 +00:00
Daniel Verkamp
46844e7131 rutabaga_gfx: ignore unused consts warning
Annotate the THREAD_SYNC flags with allow statements so that clippy does
not complain about them.

BUG=None
TEST=bin/clippy

Change-Id: I9d9d5d5b2a5f51e3ce49c36dd7528ad677aeb30e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2638139
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-01-21 02:06:46 +00:00
Dennis Kempin
0797a55462 Framework for extended integration tests
This CL expands the existing boot.rs test to not just boot a kernel
but also provide a debian-based rootfs and a special init binary
that is used to communicate between test code and the guest VM.

The delegate binary listens for commands on /dev/ttyS1 and returns
the stdout of the executed command.
This allows the test code to setup pipes for the serial device to
issue commands in the client and receive the command output, which
provides a good foundation for tests of basic functionality without
the need to pass test binary code into the guest.

The integration tests will pull a prebuilt kernel and rootfs image
from cloud storage unless local files are specified via ENV variables.

The integration_tests/guest_under_test directory contains the files
needed to build and upload those prebuilts.

BUG=b:172926609
TEST=This is a test.

Cq-Depend: chromium:2551073
Change-Id: Iffb88a146a13d1b6ed7250df1b487bd87a5599d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2536831
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
2021-01-20 17:48:10 +00:00
Dennis Kempin
191b95b5d7 native and aarch64 cross-compile containers
This CL adds the foundation for running tests consistently
in Kokoro and locally, for both x86 and aarch64.

The crosvm_builder is similar to the original image from
docker/crosvm.Dockerfile.
The main difference is that ChromeOS dependencies are not
compiled into the container, but built at runtime.

The crosvm_aarch64_builder installs the build enviornment
to cross-compile crosvm for aarch64. The tests are run
with user-space emulation using qemu-aarch64-static.

See ci/README.md for instructions on how to use these
builders.

Tests on aarch64 cannot all be run using user-space
emulation. We will need a VM to pass all smoke tests,
this work is tracked in b/177228167.

BUG=b:177133814
TEST=Tested by running
./ci/builder bin/smoke_test
./ci/builder cargo test
./ci/aarch64_builder cargo build
./ci/aarch64_builder cargo test -p tempfile

Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-20 17:41:27 +00:00
Junichi Uekawa
c7aa52317b crosvm: Add a error message signature.
Make it stand out in logs and easier to `grep` for the message in
automated scripts such as tests.

BUG=b:152259067
TEST=Introduce failure in crosvm and try booting ARCVM

Change-Id: I0ca76f8a68eb650c95e072224d0a9d019696ac6c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2413914
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
2021-01-20 11:57:09 +00:00
Chirantan Ekbote
d7bccca9a6 cros_async: Make select a future combinator
This should simplify the implementation.

BUG=none
TEST=unit tests

Change-Id: Ie2b59e7f69fc765cd7edd8dc077797ce9b8a76e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571150
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:32 +00:00
Chirantan Ekbote
12e20d2d12 cros_async: Make complete a future combinator
This will hopefully simplify the implementation.

BUG=none
TEST=unit tests

Change-Id: Iba6f0bfd1c93393a89dddea66074f404c30202d1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2550479
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:31 +00:00
Chirantan Ekbote
5380b44aef io_uring: Use IoBufMut instead of iovec
This is ABI-compatible with iovec and comes with Send + Sync impls,
which iovec does not have.

BUG=none
TEST=unit tests

Change-Id: Ic5831f557933b1d0dc823b1f6ba991ffe57baf2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571149
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:30 +00:00
Chirantan Ekbote
984d7b5a2a data_model: Rename IoSliceMut -> IoBufMut
This should hopefully make it easier to distinguish it from
std::io::IoSliceMut.

BUG=none
TEST=unit tests

Change-Id: I0746239c23e014cdccac62f1b89ec78f4008d22c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627238
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:28 +00:00
Andrew Walbran
9cfdbd9cc0 Fix clippy errors and some warnings.
TEST=cargo test

Change-Id: I91afe78141c717a592eec5ec77acac2a500ba163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2623941
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Andrew Walbran <qwandor@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2021-01-18 15:50:25 +00:00
Chirantan Ekbote
4223a41932 cros_async: Use fcntl instead of dup
dup doesn't allow us to pass in the CLOEXEC flag.

BUG=none
TEST=unit tests

Change-Id: Ia8572fa42b853e50773c80f22ec0a598e940fe48
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2629148
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-18 11:45:38 +00:00
Jason Macnak
d8dd1c62c7 rutabaga_gfx: fix iov scope with transfer_read
Move the RutabagaIovec representing the display framebuffer out of
match's Some() condition so that it outlives the call to the
underlying renderer.

BUG=b:173630595
TEST=launch_cvd --gpu_mode=gfxstream

Change-Id: I86b2ddd0de0551bf0fbbbfebf811c39d7581f3cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627646
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Jason Macnak <natsu@google.com>
Commit-Queue: Jason Macnak <natsu@google.com>
2021-01-16 04:59:51 +00:00
Jason Macnak
eaec45bc8c rutabaga_gfx: Pass display dimensions and gl flags to gfxstream
BUG=b:173630595
TEST=launch_cvd --gpu_mode=gfxstream

Change-Id: Ie745424f1d5e18c9402066cf80c45770dcd86671
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627645
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Jason Macnak <natsu@google.com>
Commit-Queue: Jason Macnak <natsu@google.com>
2021-01-16 04:59:50 +00:00
Ram Muthiah
a81bedbf92 AARCH64 bios support
BUG=b:171352650
TEST=boot test on arm cuttlefish on rockpi host with bootloader from
https://r.android.com/1504313

Change-Id: I45841178c50207ad31ce597186643a0ae1ed52ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627445
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Ram Muthiah <rammuthiah@google.com>
Auto-Submit: Ram Muthiah <rammuthiah@google.com>
2021-01-16 00:56:32 +00:00
Daniel Verkamp
de498d14f4 fuse: supply a no-op Mapper for fuzzing
The fuzz_server interface was updated to require a Mapper; however,
fs_server fuzzer was not updated to provide one, and it does not seem
necessary since the Mapper calls don't affect control flow in the
server.  Instead, just provide a no-op Mapper inside fuzz_server.

Fixes fs_server_fuzzer build.

BUG=chromium:1166742
TEST=FEATURES=test emerge-amd64-generic crosvm # --profile=asan

Change-Id: I6f2d2c7156056976a2e5dd1d7b5a29a231608150
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2629970
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-01-15 17:14:01 +00:00
Lingfeng Yang
f35d2c43ff gfxstream: use WC memory in ASG blobs
This seems to be faster than using UC and brings perf close to or
on par with goldfish address space blobs.

BUG=b:177241396

Change-Id: I8198ac0a0510388e79753cdb22007f97dcb3568b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621997
Commit-Queue: Lingfeng Yang <lfy@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Lingfeng Yang <lfy@google.com>
2021-01-14 20:54:20 +00:00
Keiichi Watanabe
9eb640ce0d fuse: Fix build error with fuzzing cfg flag
BUG=chromium:1160986
TEST=run `RUSTFLAGS="--cfg=fuzzing" cargo build`

Change-Id: If7a4091a67bc119ae06d576fe9d6f410759c8f9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2616038
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2021-01-11 21:27:07 +00:00
Allen Webb
191444797d data_model: Ad is_empty to fix clippy warning.
warning: item `sys::IoSliceMut<'a>` has a public `len` method but no
corresponding `is_empty` method
  --> /mnt/host/source/src/platform/crosvm/data_model/src/sys.rs:23:1

Also, add .idea to the .gitignore to ignore Intelij projects.

BUG=None
TEST=(cd data_model && cargo clippy)

Change-Id: I3ce0dc49bcb75ac16bc9ae3b6eec8d34f25919c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2616806
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Allen Webb <allenwebb@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-01-09 05:31:17 +00:00
Chih-Yu Huang
43c0add2b7 virtio: video: change minimum input buffer of encoder to 1
The encoder should be able to start encoding when receiving 1 input
buffer. This CL changes the minimum input buffer from 2 to 1.

BUG=b:175745193
TEST=android.mediav2.cts.EncoderProfileLevelTest

Change-Id: I97060d0304cf41369467dc62d12bbae7e54b6b22
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2602299
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Chih-Yu Huang <akahuang@chromium.org>
2020-12-25 13:02:31 +00:00
Charles William Dick
0e3d4b647a crosvm make balloon logic more fair
Introduces new struct/impl BalloonPolicy that manages the size of the
virtio balloon.

Instead of the balloon balancing available memory between the host and
guest, available memory above the critical margin is balanced. Where
critical margin is the level that the host/guest starts reclaiming
memory or killing processes. See comments above impl BalloonPolicy for
explanation.

In addition, balloon size dependent bias is introduced to encourage
large balloons to deflate, and small balloons to inflate. This bias is
needed for multivm.LifecycleShifting.arc_host to pass, as sometimes
without the bias the balloon is inflated so much that the first App in
a streak times out when launching. The bias also reduces variance in
multivm.Lifecycle.arc_host.

BUG=b:172870597
TEST=tast run dut mutivm.Lifecycle.* multivm.LifecycleShifting.* x10

Change-Id: I241e470aece56e3dc53fe59f833a1f7f4f684299
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2596293
Tested-by: Charles Dueck <cwd@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Charles Dueck <cwd@chromium.org>
2020-12-25 11:42:55 +00:00
Chih-Yu Huang
442903bf72 virtio: video: change minimum input buffer of decoder to 1
The decoder should be able to start decoding when receiving 1 input
buffer. This CL changes the minimum input buffer from 2 to 1.

BUG=b:176266656
TEST=android.media.cts.DecoderTest#testEOSBehaviorVP8

Change-Id: I2b055b145993d35c00a07bc7081db84117e36423
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2602297
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Chih-Yu Huang <akahuang@chromium.org>
2020-12-25 03:02:09 +00:00
Keiichi Watanabe
bba763f2e6 devices: virtio: fs: Avoid allocating shared memory on ARM
Avoid allocating shared memory on ARM where DAX is unsupported.

BUG=b:147341783
TEST=tast run <kukui> vm.Virtiofs

Change-Id: I347acc2243ed08ac25c4a83309e6d2957dd473ab
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2599565
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-12-23 06:24:19 +00:00
Daniel Verkamp
bccc20d963 kokoro: update docs for main branch
BUG=None
TEST=read me

Change-Id: Id3da7661e905edf6a4874849d42541bda835fbf6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2597896
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-21 20:19:20 +00:00
Gurchetan Singh
028b8044bc crosvm: clearer naming about GPU features
It's possible to compile the gpu device without virgl_renderer.
In fact, in many instances, this may be required.

This builds the gpu device default, but only with --gpu do I see
/dev/dri/renderN128, so this should be safe.

BUG=b:173630595
TEST=compile and run

Cq-Depend: chromium:2592111
Change-Id: I5fbf2de8a2f818a9ca2e5ac4a1a02c7797cff927
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2592089
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2020-12-17 01:18:00 +00:00
Daniel Verkamp
fb54f3ccd9 docs: update chromium docs URLs to HEAD branch
This makes the links work regardless of the name of the main branch.

BUG=None
TEST=Navigate to the URLs in a browser

Change-Id: I2daaa077e77e923313fecb23e6afde2e951d82a9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2596050
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-16 22:53:27 +00:00
Keiichi Watanabe
3267e022a5 Cargo.lock: Remove unnecessary dependency
Remove a dependency which was accidentally added at CL:2108315.

BUG=none
TEST=kokoro

Change-Id: I91402dd4b40188297598b5446629a7d0b1d95015
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2594951
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-16 22:11:21 +00:00
Keiichi Watanabe
eefe7fb19e devices: virtio: fs: DAX based shared memory support
Support virtio-fs's DAX (direct memory access) operation which allows the guest
to directly access file pages.
Specifically, FUSE_SETUP_MAPPING and FUSE_REMOVE_MAPPING operations are
supported.

This option can be used by specifing `dax` option when mount a file system in
the guest.

The DAX optoin improved file I/O performance in most cases.
In Fio tests, both of read and write score were improved by 1.3-14x depending on
test cases.
In Blogbench tests, which create many small files, DAX improved the write score
by 1.5x while the read score was reduced to ~25% (20391 -> 4593).

Here is an excerpt of results:
Fio
* seq_read:     10.2x (143528 -> 1464911)
* seq_write:     3.3x (61253 -> 896791)
* rand_read:    11.6x (138753 -> 1612739)
* rand_write:   14.6x (61253 -> 896791)
* surfing_read:  1.3x (98473 -> 127907)
* surfing_write: 1.3x (83309 -> 108089)

Blogbench
* read:  0.23x (20391 -> 4593)
* write: 1.50x (248 -> 373)

BUG=b:147341783
TEST=Run vm.{Blogbench, Fio} with CL:2291856

Change-Id: I4a47c601412ed32d926de6304337e1594252d258
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2108315
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-12-16 06:03:54 +00:00