Commit graph

1933 commits

Author SHA1 Message Date
Chirantan Ekbote
4a86ce877a cros_async: Simplify uring executor drop impl
Closing the uring fd should release the resources borrowed by the kernel
so there's no reason to call io_uring_enter to drive any pending IO to
completion.  Doing so may end up blocking the thread indefinitely.

Instead just make sure that the URingContext is dropped before anything
else.  Rust's drop order guarantees that struct fields are dropped in
the order of declaration so this just means putting it first.

Also, since the RawExecutor is wrapped in an Arc it may end up being
dropped from a different thread than the one that called run() or
run_until().  We know there are no other references so just clear the
cached thread_id so that we don't panic in the drop impl.  We don't
currently have any users that want to run the executor concurrently on
multiple threads so we'll just punt that problem until we actually need
to deal with it.

BUG=none
TEST=unit tests

Change-Id: Icf6a23fc433128dd00adbd56a715dbae24cd8ea2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643845
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-26 09:15:13 +00:00
Chirantan Ekbote
d1e8f146b1 cros_async: Replace notify_task with IORING_OP_NOP
IORING_OP_NOP is a no-op that does no IO and can be used to measure the
performance of the io_uring subsystem or to wake up a thread currently
blocked inside an io_uring_enter syscall. Use it instead of keeping a
separate eventfd.

BUG=none
TEST=unit tests

Change-Id: I3b3e93c653e1db747c90456c1d93abb979c25d34
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643844
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
2021-01-26 09:15:12 +00:00
Chirantan Ekbote
1fcc8a8f3c io_uring: Make UringContext Sync
Rather than requiring callers to wrap the entire UringContext in a big
lock, make UringContext Sync by internally using fine-grained locks:

* The submit queue is wrapped with a Mutex so that threads may still add
  operations to the submit queue even when a thread is blocked
  inside an io_uring_enter call.
* The completion queue internally uses a Mutex around the pending op
  data and completed count so that two threads don't end up trying to
  remove the same operation from the completed queue.

With this we can enable the await_uring_from_poll test.  This test
uncovered missing wakeups in the case where a uring operation is added
from one thread while the main uring thread is blocked inside an
io_uring_enter syscall. To deal with this, we call submit() whenever the
pending operation is polled and not yet submitted.

BUG=none
TEST=unit tests

Change-Id: I4c7dec65c03b7b10014f4a84d2cd16fe8758ea72
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643842
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-01-26 09:15:11 +00:00
Chirantan Ekbote
1e9974ff3e cros_async: Add the WeakWake trait
This is like future::task::ArcWake but uses Weak<T> instead of Arc<T>.
This prevents a circular reference where the RunnableQueue would hold an
active reference to the executor (via its waker) while the executor
owned the RunnableQueue, leading to memory leaks.

With this the executor should only be using weak references internally
and the only strong references should be in the public *Executor type.

Also fix the tests that were broken by this change.

BUG=none
TEST=unit tests

Change-Id: I3e9c007d533e154b2ad9d1e4d56a692f65da6aa0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643841
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: Chirantan Ekbote <chirantan@chromium.org>
2021-01-25 08:52:27 +00:00
Chirantan Ekbote
f1ce859903 cros_async: Refactor executors
There are a lot of changes in this one but these are the high-level
points:

* Both executors now support non-fd futures and it's now possible to
  start a poll operation on one thread and then await the result inside
  a UringExecutor on another thread.  The reverse doesn't work yet but
  will once we make UringContext sync.
* A few layers of indirection have been removed so hopefully both the
  implementation and the interface are simpler.
* The thread local magic is gone in favor of directly calling methods on
  an executor.  Executor handles can be cheaply cloned to make this
  easier.
* Heap allocations are limited to the spawn and spawn_local methods so
  it's possible to completely avoid heap allocations if callers only use
  the run_until method.
* The IoSource, Executor, FutureList traits are gone.

BUG=none
TEST=unit tests

Cq-Depend: chromium:2629068
Change-Id: I86053007929c36da66f3b2499cdefce0b5e9a180
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571401
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2021-01-25 08:52:27 +00:00
Ram Muthiah
890ae4cdb8 Relocate FDT to beginning of RAM
This change serves two purposes. The FDT needs to be in a location
accessible by 32 bit devices. Secondly the FDT's location was dependent
on the amount of RAM allocated to the guest prior to this change which
made passing it's location to the BIOS challenging.

By moving the FDT to a fixed location within the 32 bit accesible memory
boundary, these problems have been fixed.

BUG=b:177926450
TEST=local boot of android virtual device on ARM host
Change-Id: I89e481f66ddc8ffcf0e86f00fbfa92d407c48736
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2641126
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Ram Muthiah <rammuthiah@google.com>
Commit-Queue: Ram Muthiah <rammuthiah@google.com>
2021-01-23 02:22:28 +00:00
Dennis Kempin
b40ec47309 Make Kokoro happy
A few things happened:

- A recent minigbm change broke compilation for debian, so
  I am pinning the commit SHA for the Dockerfile.
- Newer rust versions come with new clippy errors, which I will
  have to look at in a separate change. For now I am pinning our
  rust version.
- Some repos have transitioned to a main branch. Fixed our uprev
  script to handle those.
- Renamed SYSROOT so we won't confuse pkg-config.

Now Kokoro is finally happy again!

BUG=None
TEST=wrapped_smoke_tests

Change-Id: I76294abe35fd84b305124158b3942a321651982d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2643779
Tested-by: Dennis Kempin <denniskempin@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-01-22 22:32:37 +00:00
Chia-I Wu
d562b1a788 gpu: bind mount /usr/share/vulkan for vulkan loader
The loader scans /usr/share/vulkan/icd.d for drivers.

BUG=b:176158397
TEST=vkcube

Change-Id: I63607148a534980479d1fd3aed8ea7b790c46c3a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2634994
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Chia-I Wu <olv@google.com>
Commit-Queue: Chia-I Wu <olv@google.com>
2021-01-22 13:28:34 +00:00
Chia-I Wu
694c53031a seccomp: gpu: allow vulkan loader and intel anv
BUG=b:176158397
TEST=vkcube

Change-Id: I11b5a97929fbb2c0328aca7d54b5882823b67209
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2634993
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Chia-I Wu <olv@google.com>
Commit-Queue: Chia-I Wu <olv@google.com>
2021-01-22 13:28:33 +00:00
Chia-I Wu
186908d641 virgl: use virgl_renderer_context_create_with_flags
When context_init is non-zero, pass it to virglrenderer by calling
virgl_renderer_context_create_with_flags.

BUG=b:176158397
TEST=vkcube

Change-Id: If9b8c60be7e6f1ef643077bd6a94aed0235a30f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2634992
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Chia-I Wu <olv@google.com>
Commit-Queue: Chia-I Wu <olv@google.com>
2021-01-22 13:28:32 +00:00
Dennis Kempin
2118d6faa3 Test Runner for crosvm
The ./run_tests script will select which tests to run on the host.
Unfortunately a lot of our tests require dependencies or access
to certain kernel devices. We cannot run all tests on all hosts.

The test runner works with the CI builders:

To run all x86 tests:
./ci/builder ./run_tests --all

To run tests on aarch64:
./ci/aarch64_builder ./run_tests --all

Or to just run tests that can be run on a standard debian buster
system without obscure chromeos dependencies:

./run_tests

Eventually, the test runner should run tests via the VM of the
aarch64_builder, currently only those tests that can be run with
user-space emulation are executed.

Currently some tests remain disabled in the runner, as they do not
pass in the CI builders yet. These will be enabled as the builders
are set up to allow for them to be run.

BUG=b:173833661
TEST=Ran the above commands.

Change-Id: Id69027793348f4d3f20adf5333d8bdfff1aa9c8b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633889
Tested-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-22 11:05:57 +00:00
Michael Hoyle
bc94c97300 vm_memory: Update memfd variable name
Update memfd -> shm to better match downstream.

BUG=b:176188689
TEST=./build_test

Change-Id: I32ba56fe89938ca58130ba02878bad2cb1f5012e
Merged-In: I7e8c91fec76babd2f6f10f9f739e1d7a181b65e7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633888
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2021-01-22 05:02:00 +00:00
Daniel Verkamp
c7236f2cf1 seccomp: allow USBDEVFS_CLEAR_HALT
This USB ioctl is used in usb_util::Device::clear_halt(), but it was not
allowed in the seccomp policy.

BUG=chromium:1167286
TEST=Attach Keyspan USA-19H USB serial adapter to Crostini

Change-Id: I625cde121a0a248046e476eecd732a98530811dc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2633824
Reviewed-by: Matthew Blecker <matthewb@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Matthew Blecker <matthewb@chromium.org>
Commit-Queue: Matthew Blecker <matthewb@chromium.org>
2021-01-22 05:00:05 +00:00
Daniel Verkamp
46844e7131 rutabaga_gfx: ignore unused consts warning
Annotate the THREAD_SYNC flags with allow statements so that clippy does
not complain about them.

BUG=None
TEST=bin/clippy

Change-Id: I9d9d5d5b2a5f51e3ce49c36dd7528ad677aeb30e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2638139
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-01-21 02:06:46 +00:00
Dennis Kempin
0797a55462 Framework for extended integration tests
This CL expands the existing boot.rs test to not just boot a kernel
but also provide a debian-based rootfs and a special init binary
that is used to communicate between test code and the guest VM.

The delegate binary listens for commands on /dev/ttyS1 and returns
the stdout of the executed command.
This allows the test code to setup pipes for the serial device to
issue commands in the client and receive the command output, which
provides a good foundation for tests of basic functionality without
the need to pass test binary code into the guest.

The integration tests will pull a prebuilt kernel and rootfs image
from cloud storage unless local files are specified via ENV variables.

The integration_tests/guest_under_test directory contains the files
needed to build and upload those prebuilts.

BUG=b:172926609
TEST=This is a test.

Cq-Depend: chromium:2551073
Change-Id: Iffb88a146a13d1b6ed7250df1b487bd87a5599d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2536831
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
2021-01-20 17:48:10 +00:00
Dennis Kempin
191b95b5d7 native and aarch64 cross-compile containers
This CL adds the foundation for running tests consistently
in Kokoro and locally, for both x86 and aarch64.

The crosvm_builder is similar to the original image from
docker/crosvm.Dockerfile.
The main difference is that ChromeOS dependencies are not
compiled into the container, but built at runtime.

The crosvm_aarch64_builder installs the build enviornment
to cross-compile crosvm for aarch64. The tests are run
with user-space emulation using qemu-aarch64-static.

See ci/README.md for instructions on how to use these
builders.

Tests on aarch64 cannot all be run using user-space
emulation. We will need a VM to pass all smoke tests,
this work is tracked in b/177228167.

BUG=b:177133814
TEST=Tested by running
./ci/builder bin/smoke_test
./ci/builder cargo test
./ci/aarch64_builder cargo build
./ci/aarch64_builder cargo test -p tempfile

Change-Id: Iffffcf48894787dd72fff894af351fdaced0b429
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621994
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-01-20 17:41:27 +00:00
Junichi Uekawa
c7aa52317b crosvm: Add a error message signature.
Make it stand out in logs and easier to `grep` for the message in
automated scripts such as tests.

BUG=b:152259067
TEST=Introduce failure in crosvm and try booting ARCVM

Change-Id: I0ca76f8a68eb650c95e072224d0a9d019696ac6c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2413914
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: Junichi Uekawa <uekawa@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Junichi Uekawa <uekawa@chromium.org>
2021-01-20 11:57:09 +00:00
Chirantan Ekbote
d7bccca9a6 cros_async: Make select a future combinator
This should simplify the implementation.

BUG=none
TEST=unit tests

Change-Id: Ie2b59e7f69fc765cd7edd8dc077797ce9b8a76e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571150
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:32 +00:00
Chirantan Ekbote
12e20d2d12 cros_async: Make complete a future combinator
This will hopefully simplify the implementation.

BUG=none
TEST=unit tests

Change-Id: Iba6f0bfd1c93393a89dddea66074f404c30202d1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2550479
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:31 +00:00
Chirantan Ekbote
5380b44aef io_uring: Use IoBufMut instead of iovec
This is ABI-compatible with iovec and comes with Send + Sync impls,
which iovec does not have.

BUG=none
TEST=unit tests

Change-Id: Ic5831f557933b1d0dc823b1f6ba991ffe57baf2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571149
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:30 +00:00
Chirantan Ekbote
984d7b5a2a data_model: Rename IoSliceMut -> IoBufMut
This should hopefully make it easier to distinguish it from
std::io::IoSliceMut.

BUG=none
TEST=unit tests

Change-Id: I0746239c23e014cdccac62f1b89ec78f4008d22c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627238
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-19 11:59:28 +00:00
Andrew Walbran
9cfdbd9cc0 Fix clippy errors and some warnings.
TEST=cargo test

Change-Id: I91afe78141c717a592eec5ec77acac2a500ba163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2623941
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Andrew Walbran <qwandor@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2021-01-18 15:50:25 +00:00
Chirantan Ekbote
4223a41932 cros_async: Use fcntl instead of dup
dup doesn't allow us to pass in the CLOEXEC flag.

BUG=none
TEST=unit tests

Change-Id: Ia8572fa42b853e50773c80f22ec0a598e940fe48
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2629148
Auto-Submit: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-01-18 11:45:38 +00:00
Jason Macnak
d8dd1c62c7 rutabaga_gfx: fix iov scope with transfer_read
Move the RutabagaIovec representing the display framebuffer out of
match's Some() condition so that it outlives the call to the
underlying renderer.

BUG=b:173630595
TEST=launch_cvd --gpu_mode=gfxstream

Change-Id: I86b2ddd0de0551bf0fbbbfebf811c39d7581f3cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627646
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Jason Macnak <natsu@google.com>
Commit-Queue: Jason Macnak <natsu@google.com>
2021-01-16 04:59:51 +00:00
Jason Macnak
eaec45bc8c rutabaga_gfx: Pass display dimensions and gl flags to gfxstream
BUG=b:173630595
TEST=launch_cvd --gpu_mode=gfxstream

Change-Id: Ie745424f1d5e18c9402066cf80c45770dcd86671
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627645
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Jason Macnak <natsu@google.com>
Commit-Queue: Jason Macnak <natsu@google.com>
2021-01-16 04:59:50 +00:00
Ram Muthiah
a81bedbf92 AARCH64 bios support
BUG=b:171352650
TEST=boot test on arm cuttlefish on rockpi host with bootloader from
https://r.android.com/1504313

Change-Id: I45841178c50207ad31ce597186643a0ae1ed52ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2627445
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Ram Muthiah <rammuthiah@google.com>
Auto-Submit: Ram Muthiah <rammuthiah@google.com>
2021-01-16 00:56:32 +00:00
Daniel Verkamp
de498d14f4 fuse: supply a no-op Mapper for fuzzing
The fuzz_server interface was updated to require a Mapper; however,
fs_server fuzzer was not updated to provide one, and it does not seem
necessary since the Mapper calls don't affect control flow in the
server.  Instead, just provide a no-op Mapper inside fuzz_server.

Fixes fs_server_fuzzer build.

BUG=chromium:1166742
TEST=FEATURES=test emerge-amd64-generic crosvm # --profile=asan

Change-Id: I6f2d2c7156056976a2e5dd1d7b5a29a231608150
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2629970
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2021-01-15 17:14:01 +00:00
Lingfeng Yang
f35d2c43ff gfxstream: use WC memory in ASG blobs
This seems to be faster than using UC and brings perf close to or
on par with goldfish address space blobs.

BUG=b:177241396

Change-Id: I8198ac0a0510388e79753cdb22007f97dcb3568b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2621997
Commit-Queue: Lingfeng Yang <lfy@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Lingfeng Yang <lfy@google.com>
2021-01-14 20:54:20 +00:00
Keiichi Watanabe
9eb640ce0d fuse: Fix build error with fuzzing cfg flag
BUG=chromium:1160986
TEST=run `RUSTFLAGS="--cfg=fuzzing" cargo build`

Change-Id: If7a4091a67bc119ae06d576fe9d6f410759c8f9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2616038
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2021-01-11 21:27:07 +00:00
Allen Webb
191444797d data_model: Ad is_empty to fix clippy warning.
warning: item `sys::IoSliceMut<'a>` has a public `len` method but no
corresponding `is_empty` method
  --> /mnt/host/source/src/platform/crosvm/data_model/src/sys.rs:23:1

Also, add .idea to the .gitignore to ignore Intelij projects.

BUG=None
TEST=(cd data_model && cargo clippy)

Change-Id: I3ce0dc49bcb75ac16bc9ae3b6eec8d34f25919c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2616806
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Allen Webb <allenwebb@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-01-09 05:31:17 +00:00
Chih-Yu Huang
43c0add2b7 virtio: video: change minimum input buffer of encoder to 1
The encoder should be able to start encoding when receiving 1 input
buffer. This CL changes the minimum input buffer from 2 to 1.

BUG=b:175745193
TEST=android.mediav2.cts.EncoderProfileLevelTest

Change-Id: I97060d0304cf41369467dc62d12bbae7e54b6b22
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2602299
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Chih-Yu Huang <akahuang@chromium.org>
2020-12-25 13:02:31 +00:00
Charles William Dick
0e3d4b647a crosvm make balloon logic more fair
Introduces new struct/impl BalloonPolicy that manages the size of the
virtio balloon.

Instead of the balloon balancing available memory between the host and
guest, available memory above the critical margin is balanced. Where
critical margin is the level that the host/guest starts reclaiming
memory or killing processes. See comments above impl BalloonPolicy for
explanation.

In addition, balloon size dependent bias is introduced to encourage
large balloons to deflate, and small balloons to inflate. This bias is
needed for multivm.LifecycleShifting.arc_host to pass, as sometimes
without the bias the balloon is inflated so much that the first App in
a streak times out when launching. The bias also reduces variance in
multivm.Lifecycle.arc_host.

BUG=b:172870597
TEST=tast run dut mutivm.Lifecycle.* multivm.LifecycleShifting.* x10

Change-Id: I241e470aece56e3dc53fe59f833a1f7f4f684299
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2596293
Tested-by: Charles Dueck <cwd@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Charles Dueck <cwd@chromium.org>
2020-12-25 11:42:55 +00:00
Chih-Yu Huang
442903bf72 virtio: video: change minimum input buffer of decoder to 1
The decoder should be able to start decoding when receiving 1 input
buffer. This CL changes the minimum input buffer from 2 to 1.

BUG=b:176266656
TEST=android.media.cts.DecoderTest#testEOSBehaviorVP8

Change-Id: I2b055b145993d35c00a07bc7081db84117e36423
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2602297
Tested-by: Chih-Yu Huang <akahuang@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Chih-Yu Huang <akahuang@chromium.org>
2020-12-25 03:02:09 +00:00
Keiichi Watanabe
bba763f2e6 devices: virtio: fs: Avoid allocating shared memory on ARM
Avoid allocating shared memory on ARM where DAX is unsupported.

BUG=b:147341783
TEST=tast run <kukui> vm.Virtiofs

Change-Id: I347acc2243ed08ac25c4a83309e6d2957dd473ab
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2599565
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-12-23 06:24:19 +00:00
Daniel Verkamp
bccc20d963 kokoro: update docs for main branch
BUG=None
TEST=read me

Change-Id: Id3da7661e905edf6a4874849d42541bda835fbf6
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2597896
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-21 20:19:20 +00:00
Gurchetan Singh
028b8044bc crosvm: clearer naming about GPU features
It's possible to compile the gpu device without virgl_renderer.
In fact, in many instances, this may be required.

This builds the gpu device default, but only with --gpu do I see
/dev/dri/renderN128, so this should be safe.

BUG=b:173630595
TEST=compile and run

Cq-Depend: chromium:2592111
Change-Id: I5fbf2de8a2f818a9ca2e5ac4a1a02c7797cff927
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2592089
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2020-12-17 01:18:00 +00:00
Daniel Verkamp
fb54f3ccd9 docs: update chromium docs URLs to HEAD branch
This makes the links work regardless of the name of the main branch.

BUG=None
TEST=Navigate to the URLs in a browser

Change-Id: I2daaa077e77e923313fecb23e6afde2e951d82a9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2596050
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-16 22:53:27 +00:00
Keiichi Watanabe
3267e022a5 Cargo.lock: Remove unnecessary dependency
Remove a dependency which was accidentally added at CL:2108315.

BUG=none
TEST=kokoro

Change-Id: I91402dd4b40188297598b5446629a7d0b1d95015
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2594951
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-16 22:11:21 +00:00
Keiichi Watanabe
eefe7fb19e devices: virtio: fs: DAX based shared memory support
Support virtio-fs's DAX (direct memory access) operation which allows the guest
to directly access file pages.
Specifically, FUSE_SETUP_MAPPING and FUSE_REMOVE_MAPPING operations are
supported.

This option can be used by specifing `dax` option when mount a file system in
the guest.

The DAX optoin improved file I/O performance in most cases.
In Fio tests, both of read and write score were improved by 1.3-14x depending on
test cases.
In Blogbench tests, which create many small files, DAX improved the write score
by 1.5x while the read score was reduced to ~25% (20391 -> 4593).

Here is an excerpt of results:
Fio
* seq_read:     10.2x (143528 -> 1464911)
* seq_write:     3.3x (61253 -> 896791)
* rand_read:    11.6x (138753 -> 1612739)
* rand_write:   14.6x (61253 -> 896791)
* surfing_read:  1.3x (98473 -> 127907)
* surfing_write: 1.3x (83309 -> 108089)

Blogbench
* read:  0.23x (20391 -> 4593)
* write: 1.50x (248 -> 373)

BUG=b:147341783
TEST=Run vm.{Blogbench, Fio} with CL:2291856

Change-Id: I4a47c601412ed32d926de6304337e1594252d258
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2108315
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-12-16 06:03:54 +00:00
Daniel Verkamp
76ab3a4eb0 docker: update checkout_commits.env
Update all git repositories to the latest after verifying the tests
still pass.

BUG=None
TEST=docker/build_crosvm_base.sh && docker/wrapped_smoke_test.sh

Change-Id: Ie5e93ce76c0a180dea36e3909b16de8900b6461a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2593458
Reviewed-by: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-12-16 02:31:32 +00:00
Keiichi Watanabe
853ecdb355 clippy: Fix unnecessary-lazy-evaluations
BUG=none
TEST=bin/clippy

Change-Id: I2d0586bbb16619d9a692907235e8318edf3de9a0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2592003
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-12-15 18:01:54 +00:00
Keiichi Watanabe
39d9542bc0 clippy: Update doc comment to avoid a bug of clippy 1.48.0
clippy 1.48.0 has a bug in string parsing in doc comments.
https://github.com/rust-lang/rust-clippy/issues/6022

To avoid this issue, update to use double-quotation instead of single-quotation
inside a code block in a doc comment.

Also, this fixes wrong examples of commands there at the same time.

BUG=none
TEST=bin/clippy

Change-Id: I90e5699f6d4e839304f9d4e05e5c7b21467744cd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2592001
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-12-15 18:01:52 +00:00
Camilo Jacomet
a9bfcb485c Fixed Clippy or_fun_call warnings in devices/
BUG=chromium:1111728
TEST=cargo clippy -- -A clippy::all -W clippy::or_fun_call passes under
devices directory

Change-Id: I4c9e079df98b50e55a5489cd925fb7b621caf4dd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2587191
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: Camilo Jacomet <cjacomet@google.com>
Commit-Queue: Camilo Jacomet <cjacomet@google.com>
2020-12-15 05:15:00 +00:00
Camilo Jacomet
48d1729e2f Fixed or_fun_call warnings in kvm/
BUG=chromium:1111728
TEST=cargo clippy -- -A clippy::all -W clippy::or_fun_call passes under kvm
directory

Change-Id: I7677856f2944f3d4e1c92d2f6ec75763faa5ee42
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2582942
Tested-by: Camilo Jacomet <cjacomet@google.com>
Commit-Queue: Camilo Jacomet <cjacomet@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-12-15 04:07:40 +00:00
David Staessens
74c2381d42 crosvm: Add support for forcing keyframes to the crosvm encoder.
This CL adds support for VIRTIO_VIDEO_CONTROL_FORCE_KEYFRAME to the
crosvm encoder.

BUG=b:161498590,b:174444769
TEST=tast run DUT arc.VideoEncodeAccel.h264_192p_i420_vm

Change-Id: Ibe0c7d2aa7b41c6a0168a7aa312d6996cc22ef23
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2567308
Tested-by: David Staessens <dstaessens@chromium.org>
Commit-Queue: David Staessens <dstaessens@chromium.org>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
2020-12-14 03:25:01 +00:00
Camilo Jacomet
e08b929c25 Fixed redundant_closure warnings in kvm/
BUG=chromium:1111728
TEST=cargo clippy -- -A clippy::all -W clippy::redundant_closure passes
under kvm/

Change-Id: I3b5e03a54f9a87d1b6b905d7d8553bf2f30d9aaa
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2582946
Tested-by: Camilo Jacomet <cjacomet@google.com>
Commit-Queue: Camilo Jacomet <cjacomet@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-12-11 20:35:35 +00:00
Colin Downs-Razouk
2bf3202d63 x86_64: add integration test case
This test was useful for debugging issues with the hypervisor
abstraction layer while it was in development. It's similar to some of
the kvm integration tests, but runs the x86_64 setup functions. It has
some commented out lines for having this test load a real kernel and/or
ramdisk, which can also be useful for debugging boot problems.

RESTRICT_AUTOMERGE
BUG=b:175025264
TEST=cargo test -p x86_64

Change-Id: If5b89fe48d34db50fb962382032881e4e588db6e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2579896
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Colin Downs-Razouk <colindr@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-12-11 20:22:25 +00:00
Gurchetan Singh
6ea1f74a70 gpu_display: display_wl: wait for surface to be configured
This error was observed:

wl_surface@5: error 3: buffer committed to unconfigured xdg_surface

Shout out to bugaevc for the diagnosis and fix:

"In addition to setting up the listeners, we have also made two
important changes.  First, we call wl_surface.commit right away,
prior to attaching any buffer to it. This causes the compositor to
issue the appropriate configure events. We wait for them to be
received and handled with an additional wl_display_roundtrip() before
we attach the buffer."

https://bugaevc.gitbooks.io/writing-wayland-clients/content/beyond-the-black-square/xdg-shell.html

BUG=b:150239451
TEST=boot VM with wayland display, still doesn't work completely
     correctly due to lack of {import, attach, detach}_event_device
     implementations

Change-Id: I06fac4f8540ce410c20a367d1e5dbe3fc08c4f10
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2583164
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-12-10 22:45:08 +00:00
David Staessens
911a0de1bf crosvm: Expand the list of supported encode levels.
Currently only H.264 level 1 is reported to the virto encoder when the
supported H.264 levels are queried. This CL expands the list to contain
all levels up to 5.1 which are supported by most devices, as the VEA
interface currently has no way to query the maximum supported H.264
level.

BUG=b:174967472
TEST=android.media.cts.MediaRecorderTest#testProfileAvcBaselineLevel1

Change-Id: I960f9176fa00180da6f71bcdad558da06e8247c1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2574583
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: David Staessens <dstaessens@chromium.org>
Commit-Queue: David Staessens <dstaessens@chromium.org>
2020-12-10 09:58:22 +00:00
David Staessens
f16084de9e crosvm: Apply framerate requested on OUTPUT queue to CAPTURE queue.
Currently only the framerate requested on the CAPTURE queue is used to
configure the Chrome encoder's framerate. But according to the V4L2
standard setting the framerate on the OUTPUT queue should also set the
framerate on the CAPTURE queue to the same value.

BUG=b:173668157
TEST=android.media.cts.MediaRecorderTest#testProfileAvcBaselineLevel1

Change-Id: I0753d10d73d52ce4f17fd9bc230d4fa2f06a1b30
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2570833
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: David Staessens <dstaessens@chromium.org>
Commit-Queue: David Staessens <dstaessens@chromium.org>
2020-12-10 04:43:24 +00:00