Commit graph

260 commits

Author SHA1 Message Date
Chirantan Ekbote
a00991cd84 Replace dup with fcntl(F_DUPFD_CLOEXEC)
Fds created via dup don't share file descriptor flags with the original
fd, which means that they don't have the FD_CLOEXEC flag set.  Use
fcntl(F_DUPFD_CLOEXEC) so that this flag gets set for the duplicated fds
as well.

BUG=none
TEST=unit tests

Change-Id: Ib471cf40acac1eacf72969ba45247f50b349ed58
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2809687
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-04-15 10:34:04 +00:00
Noah Gold
9af97d72fa [base] update/clean mmap interfaces.
This CL addresses some minor issues with the existing interface:
1. from_descriptor is too generic for some platforms that require
   special handling for file/File backed mappings.
2. Nearly all call sites pass either File or SharedMemory. Now
   we just have from_ methods for those types to preserve type
   information.
3. Other platforms require additional fields in MemoryMapping, so a
   tuple struct no longer makes sense.
4. The mmap syscall error message was misleading as we use it for more
   than just the mmap syscall.

BUG=None
TEST=builds

Change-Id: I74c41bad52bb81880a11231cd18f47e233548a24
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2815614
Reviewed-by: Udam Saini <udam@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
2021-04-15 02:10:35 +00:00
Chirantan Ekbote
4b47aa7105 fs: Fix device socket
validate_raw_fd assumes that the fd passed in was not created by crosvm
and returns EBADF if it sees that the fd has the FD_CLOEXEC flag set.
We can't use it for fds created by the fs device since those do have
that flag set.

We're already taking a `&dyn AsRawFd` as the parameter so just assume
it's valid and clone it directly since there's no safe way to create an
invalid one.

BUG=none
TEST=vm.Fio.virtiofs_dax* tests are no longer failing

Change-Id: I10d9752e0960143fb58a63d2b76f64d34ec464d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2809686
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2021-04-09 08:04:30 +00:00
Allen Webb
53a5d09dd1 sys_util: Migrate code from libchromeos::linux.
BUG=chromium:1193155
TEST=emerge-${BOARD} dev-rust/sys_util

Cq-Depend: chromium:2788637
Change-Id: Ifabac09d2ebf39387cf96ebb1571d70e77eedc13
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2789053
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Allen Webb <allenwebb@google.com>
2021-04-05 21:22:49 +00:00
Zach Reizner
48e1e52928 base: add tube module
Tube is the replacement for MsgSocket and related types and traits.

TEST=run_tests
BUG=b:176847362

Change-Id: I290279a714eb04c5cc6f2aef15ba7c61c708ab08
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2726980
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-04-01 01:59:29 +00:00
Allen Webb
44c728c123 sys_util: Add functionality to signals.
This refactors register_signal_handler to take a callback with the
correct parameter (c_int).

It adds functionality for clearing signal handlers, checking if a signal
handler is defined, and waiting for a signal. As part of this work a
helper function was added to create libc::timespec structs from
std::time::Duration, and get the max Duration that can be represented by
libc::timespec.

BUG=None
TEST=cargo test

Cq-Depend: chromium:2782855, chromium:2782856
Change-Id: Id531d9aaeeeb65f0f6f9b64bed4aba024a2328cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2783043
Tested-by: Allen Webb <allenwebb@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-03-29 20:54:20 +00:00
Zach Reizner
de0c63dbcd implement Serialize and Deserialize for base and sys_util types
These types make up parts of larger messages that implemented
MsgOnSocket.

BUG=b:176847362
TEST=run_tests

Change-Id: I1f99e08f494d646ad0566eb556e2c28726d1d217
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2733207
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-03-29 19:39:52 +00:00
Zach Reizner
daceeb59c6 sys_util: add tool for raw descriptor reflection
This CL enables serde based serialize and deserialize types to interact
with a side-channel stash of descriptors. The idea is that many types
could implement Serialize and Deserialize, but they have descriptors
buried in them, such as a File or kernel object. Although these can be
serialized literally, usually the transciever of the serialized value
needs explicit access to the descriptors to send them over, such as with
Unix domain sockets.

TEST=cargo test -p base
BUG=b:176847362

Cq-Depend: chromium:2757675
Change-Id: I3273889f8d43cca761a54c531a981b7ab1ad3c03
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2576633
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2021-03-29 19:39:51 +00:00
Gurchetan Singh
6e8f33aa0a rutabaga_gfx: convert to SafeDescriptor
To be truly OS-agnostic, we need an OS-agnostic Rust wrapper over
the OS-specific handle type.  SafeDescriptor seems to be the best
option, and I hope it on crates.io in the future.

This converts virtio_gpu/rutabaga to use the SafeDescriptor handle
when practical.  minigbm still uses File in some places, since it
needs to SeekFrom(..), but minigbm is a Linux only thing anyways.

BUG=b:173630595
TEST=boot VM in 2D/3D mode

Change-Id: I18d735844d479f52d82d7976bf9b4e383b2e2252
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2779492
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2021-03-23 00:44:10 +00:00
Zach Reizner
d09392e37e uprev rust-toolchain and fix clippy warnings
Some judgement calls were made about unnecessary wrapping. Usually they
would get resolved by removing the wrapping or returning a convenient
error, but the ones that returned results for consistency with other
functions were added to the allow list.

The error handling in the usb code had a lot of unit error types which
is now a clippy lint. This was resolved by either removing the result
entirely or returning a convenient error.

The field_reassign_with_default lint is faulty and was added to the list
of supressions. This affected virtio-wayland code.

BUG=b:179277332
TEST=cargo clippy with rustc 1.50+

Change-Id: Ie812cdeaf7c42f4f2b47b1dc87f05a7c87a60f8f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2757510
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Commit-Queue: Dennis Kempin <denniskempin@google.com>
2021-03-22 21:41:07 +00:00
Dylan Reid
2320fc7a83 sys_util: sock ctrl - add send/recv flexibility
For historical reasons, crosvm switched to sending `IntoIobuf`, this
isn't implementable for a slice of slices as it would require an
allocation. The rust-vmm fork of sys_util still supports sending
`&[&[u8]]`. Adding that ability back makes it easier to use their
crates.

In addition crosvm allows receiving to slices by not iovecs, again the
opposite of the rust-vmm version. Add that ability as well.

Change-Id: I4566f232adb47a0802e39e3f6c09c03f97190eb3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2740533
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2021-03-11 08:06:27 +00:00
Keiichi Watanabe
0074b288ef sys_util: Fix clippy errors and warnings
BUG=chromium:1111728
TEST=cargo clippy --all-features --all-targets in sys_util
TEST=./test_all

Change-Id: Ib544bd1f87fa8a3244b4e6643f54eae265ef55bd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2717905
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2021-03-04 20:23:12 +00:00
Dennis Kempin
57e0d5d4dc Make Kokoro Happy
Fixes a clippy issue and disables the panic_safe test.

This test occasionally deadlocks (Maybe a 5% chance) when running
inside a test VM. I filed a bug to re-enable it.

BUG=b:179924844
TEST=kokoro passes

Change-Id: I6e9b5fc38e7bbab3fd4b2b8ba8cb5532dd14af7e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2697860
Commit-Queue: Dennis Kempin <denniskempin@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Auto-Submit: Dennis Kempin <denniskempin@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2021-02-17 07:25:38 +00:00
Dylan Reid
503c5abef6 devices: Add an asynchronous block device
This enables the use of basic disk images with async IO. A new
block_async.rs is added which mostly mirrors block, except that all
IO operations are asynchronous allowing for multiple virt queues to be
used.

The old block remains unchanged and is still used for qcow, android
sparse, and composite disks. Those should be converted to as time
allows, but this dual approach will have to do for now so ARCVM disk
performance can be properly evaluated.

fio --ioengine=libaio --randrepeat=1 --direct=1 --gtod_reduce=1
--name=test --filename=test --bs=4k --iodepth=64 --size=4G
--readwrite=randrw --rwmixread=75

desktop with nvme:

before:
READ: bw=36.2MiB/s (37.9MB/s), 36.2MiB/s-36.2MiB/s (37.9MB/s-37.9MB/s),
io=3070MiB (3219MB), run=84871-84871msec
WRITE: bw=12.1MiB/s (12.7MB/s), 12.1MiB/s-12.1MiB/s (12.7MB/s-12.7MB/s),
io=1026MiB (1076MB), run=84871-84871msec
after:
READ: bw=257MiB/s (269MB/s), 257MiB/s-257MiB/s (269MB/s-269MB/s),
io=3070MiB (3219MB), run=11964-11964msec
WRITE: bw=85.8MiB/s (89.9MB/s), 85.8MiB/s-85.8MiB/s (89.9MB/s-89.9MB/s),
io=1026MiB (1076MB), run=11964-11964msec

samus with 5.6 kernel
before:
READ: bw=55.3MiB/s (57.9MB/s), 55.3MiB/s-55.3MiB/s (57.9MB/s-57.9MB/s),
io=768MiB (805MB), run=13890-13890msec
WRITE: bw=18.5MiB/s (19.4MB/s), 18.5MiB/s-18.5MiB/s (19.4MB/s-19.4MB/s),
io=256MiB (269MB), run=13890-13890msec
after:
READ: bw=71.2MiB/s (74.7MB/s), 71.2MiB/s-71.2MiB/s (74.7MB/s-74.7MB/s),
io=3070MiB (3219MB), run=43096-43096msec
WRITE: bw=23.8MiB/s (24.0MB/s), 23.8MiB/s-23.8MiB/s (24.0MB/s-24.0MB/s),
io=1026MiB (1076MB), run=43096-43096msec

kevin with 5.6 kernel
before:
READ: bw=12.9MiB/s (13.5MB/s), 12.9MiB/s-12.9MiB/s (13.5MB/s-13.5MB/s),
io=1534MiB (1609MB), run=118963-118963msec
WRITE: bw=4424KiB/s (4530kB/s), 4424KiB/s-4424KiB/s (4530kB/s-4530kB/s),
io=514MiB (539MB), run=118963-118963msec
after:
READ: bw=12.9MiB/s (13.5MB/s), 12.9MiB/s-12.9MiB/s (13.5MB/s-13.5MB/s),
io=1534MiB (1609MB), run=119364-119364msec
WRITE: bw=4409KiB/s (4515kB/s), 4409KiB/s-4409KiB/s (4515kB/s-4515kB/s),
io=514MiB (539MB), run=119364-119364msec

eve with nvme and 5.7 kernel
before:
READ: bw=49.4MiB/s (51.8MB/s), 49.4MiB/s-49.4MiB/s (51.8MB/s-51.8MB/s),
io=3070MiB
(3219MB), run=62195-62195msec
WRITE: bw=16.5MiB/s (17.3MB/s), 16.5MiB/s-16.5MiB/s (17.3MB/s-17.3MB/s),
io=1026MiB
 (1076MB), run=62195-62195msec
after
READ: bw=125MiB/s (131MB/s), 125MiB/s-125MiB/s (131MB/s-131MB/s),
io=3070MiB (3219MB), run=24593-24593msec
WRITE: bw=41.7MiB/s (43.7MB/s), 41.7MiB/s-41.7MiB/s
(43.7MB/s-43.7MB/s), io=1026MiB (1076MB), run=24593-24593msec

rammus with 5.10 kernel
before:
READ: bw=6927KiB/s (7093kB/s), 6927KiB/s-6927KiB/s (7093kB/s-7093kB/s),
io=3070MiB (3219MB), run=453822-453822msec
WRITE: bw=2315KiB/s (2371kB/s), 2315KiB/s-2315KiB/s (2371kB/s-2371kB/s),
io=1026MiB (1076MB), run=453822-453822msec
after:
Run status group 0 (all jobs):
READ: bw=10.0MiB/s (11.5MB/s), 10.0MiB/s-10.0MiB/s (11.5MB/s-11.5MB/s),
io=3070MiB (3219MB), run=279111-279111msec
WRITE: bw=3764KiB/s (3855kB/s), 3764KiB/s-3764KiB/s (3855kB/s-3855kB/s),
io=1026MiB (1076MB), run=279111-279111msec

BUG=chromium:901139
TEST=unitests
TEST=boot a test image and run fio tests from the guest to measure speed.
TEST=start ARCVM
TEST=tast run $DUT crostini.ResizeOk.dlc_stretch_stable

Change-Id: Idb63628871d0352bd18501a69d9c1c887c37607b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2306786
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2021-02-17 04:11:55 +00:00
Andrew Walbran
9cfdbd9cc0 Fix clippy errors and some warnings.
TEST=cargo test

Change-Id: I91afe78141c717a592eec5ec77acac2a500ba163
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2623941
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Andrew Walbran <qwandor@google.com>
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2021-01-18 15:50:25 +00:00
Keiichi Watanabe
eefe7fb19e devices: virtio: fs: DAX based shared memory support
Support virtio-fs's DAX (direct memory access) operation which allows the guest
to directly access file pages.
Specifically, FUSE_SETUP_MAPPING and FUSE_REMOVE_MAPPING operations are
supported.

This option can be used by specifing `dax` option when mount a file system in
the guest.

The DAX optoin improved file I/O performance in most cases.
In Fio tests, both of read and write score were improved by 1.3-14x depending on
test cases.
In Blogbench tests, which create many small files, DAX improved the write score
by 1.5x while the read score was reduced to ~25% (20391 -> 4593).

Here is an excerpt of results:
Fio
* seq_read:     10.2x (143528 -> 1464911)
* seq_write:     3.3x (61253 -> 896791)
* rand_read:    11.6x (138753 -> 1612739)
* rand_write:   14.6x (61253 -> 896791)
* surfing_read:  1.3x (98473 -> 127907)
* surfing_write: 1.3x (83309 -> 108089)

Blogbench
* read:  0.23x (20391 -> 4593)
* write: 1.50x (248 -> 373)

BUG=b:147341783
TEST=Run vm.{Blogbench, Fio} with CL:2291856

Change-Id: I4a47c601412ed32d926de6304337e1594252d258
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2108315
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-12-16 06:03:54 +00:00
Chirantan Ekbote
6daa9a5a2c data_model: Replace as_iobuf() with AsRef and AsMut
IoSliceMut doesn't currently provide a way to access the fields of the
internal iovec. Additionally, the as_iobuf() method returns a copy of
the iovec rather than a reference, which can cause hard to diagnose
issues.

Replace the as_iobuf() method with implementations of the AsRef and
AsMut traits. This should also make it easier to hide these impls behind
a `#[cfg(unix)]` in the future.

BUG=none
TEST=unit tests

Change-Id: I58f8c5f579f3d03ec6e612a32343e8423eced8ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2571147
Reviewed-by: Noah Gold <nkgold@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2020-12-09 14:31:11 +00:00
Michael Hoyle
a596a07b0c Final major RawDescriptor transition.
This CL transitions most structs to RawDescriptor and the
associated traits if possible.

BUG=b:162363783
TEST=./build_test

Change-Id: Iabae6ac212787836d77de2b9ffb5d451421ab0dd
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2530911
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Michael Hoyle <mikehoyle@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-11-13 02:38:47 +00:00
Noah Gold
b3fca20e24 Refactor cros_async interface to avoid RawFd.
Instead of creating IoSourceExt from AsRawFd implementers, we've
switched to creating from a marker trait `IntoAsync`. This lets us use
other types like RawDescriptor easily with this crate.  By using the
marker, we also provide some type safety by requiring consumers of
IoSourceExt to declare that their type is expected to work with async
operations. This way we can provide stronger guarantees that an async
IO trait object will behave in a reasonable way.

This CL also purges the cros_async -> base and io_uring -> base
references, and provides the base types needed to add new async
primitives to base.

BUG=none
TEST=builds

Change-Id: I0b0ce6ca7938b22ae8e8fb4e604439f0292678f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2504481
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-11-10 11:40:50 +00:00
Michael Hoyle
6a0960a4a7 base: First steps towards universal RawDescriptor
This involves removing RawFd trait implementations from
Event and Timer, and resolving the echoing dependencies from
there.

Ultimately, this results mostly in changes across ioctl
(new thin layer in base), kvm, msg_on_socket, and a few other
areas. As usual, most changes are negligible.

BUG=b:162363783
TEST=./build_test

Change-Id: I47dec94666bc3430fed627b4767919c9cb4fce6f
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2462330
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-31 07:12:34 +00:00
Michael Hoyle
e392c46953 base: Add WaitContext
This is a layer above PollContext with a more generic interface.
As PollContext is used so widely, this is quite a large change
in order to accomodate the interface update, especially with the
use of RawDescriptor. In some cases this has caused an echo
of updates to RawDescriptor, which is fine because of our eventual
goal to move the whole codebase to it regardless.

Note there are a few instances of forcing the RawDescriptor update
chain to stop, ex. ioctl. This is to keep the scope of this CL
concentrated and avoid changing entire other areas.

Note that this CL leaves out a few additional pieces of work:
 - The sole usage of EpollContext over PollContext (event_loop),
which poses a bigger challenge for interface changes
 - Full PollToken renaming, which is a tiny change turned difficult
due to the unavailability of type aliases for traits.
 - Renaming certain methods which have been updated to use
RawDescriptor such as keep_fds. Some have enough dependencies that
they are worth avoiding to keep this CL pointed, but will be
addressed in future CLs to make sure the whole codebase is on the
fd->descriptor train

BUG=b:162363783
TEST=./build_test

Change-Id: Iff2cfe8f90dea55f1388f8e91bdc698e121a8e43
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2455726
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-31 07:12:30 +00:00
Michael Hoyle
3038e40a6b base: Refactor mmap to use builder pattern.
BUG=b:165423256
TEST=./build_test

Change-Id: Ia67c1a7fe29b66c9cab38476eecde8c25a55617b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2442569
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-16 07:15:24 +00:00
Zach Reizner
ab04758814 build_test: misc options, improvements, amd bug fixes
New option, --size-only, speeds up using build_test for getting release
binary size by skipping everything else. The lto flag is also added for
release builds to get a more realistic comparison.

The list of crates to test is built up automatically instead of
hard coded. To modify what gets included, empty .build_test_* files are
checked for existance. This is better than hard coding the list of
packages because it was frequently out of date.

For certain crate tests, a dynamic library that only exists in a sysroot
is required. This change includes a fix that adds the sysroot's lib
directory to the LD_LIBRARY_PATH env variable, similar to how
PKG_CONFIG_LIBDIR is modified.

TEST=build_test
BUG=None

Change-Id: I626cbcccf40035a0d29001cef7989a091848e4c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2444273
Tested-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Zach Reizner <zachr@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
2020-10-15 13:40:32 +00:00
Noah Gold
ee71c80b08 Mark MsgSocket as !Sync.
Semantically, MsgSocket is !Sync: multiple threads reading/writing to
the same end of the socket will result in U.B.. This CL brings the
syntactic definition of MsgSocket in line with its semantic meaning by
marking it as !Sync. While earlier work caught one of the other
consumers that was using a socket end in a Sync manner, this CL has
found another: the usb crate.

The fixes proposed for the usb crate are mostly for compilation's sake,
and should have little material effect on the operation of crosvm. Note
that they will assure correctness, since the send/recv for the socket
end involved are in the same critical section.

BUG=None
TEST=builds

Change-Id: I38469d9ed3228a43457b7d43879a6b4fff97b9a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2466960
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Noah Gold <nkgold@google.com>
2020-10-14 01:36:09 +00:00
Michael Hoyle
685316f0bd base: EventFd -> Event renaming
Note the CL size is large entirely due to the rename,
the changes are mostly negligible.

Also making a few small additional changes in sys_util
areas that don't need much attention in base. This includes
typedefing and adding specific imports for areas that don't
require significant interface changes.

BUG=b:162363783
TEST=./build_test

Change-Id: I4a2c9c4cdce7565806ed338e241c6b8c82c855c7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2415180
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-10-06 13:50:09 +00:00
Michael Hoyle
08d86a44a0 base: Add timer package
Very little of substance is added here, just the base boilerplate

BUG=b:162363783
TEST=./build_test

Change-Id: I2e3b3b45cf1d7234784d769b4dced31f10a8774d
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2366110
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-09-30 19:44:42 +00:00
Michael Hoyle
a7e38ab421 base: Add shared memory layer to base.
This is very largely boilerplate, but changes some interfaces
to accept different parameters, esp using Descriptor.

BUG=b:162363783
TEST=./build_test

Change-Id: I81b513c0de9448c2de505fe5d3a61a29f4eccb78
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2342591
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Michael Hoyle <mikehoyle@google.com>
2020-09-30 19:44:40 +00:00
Daniel Verkamp
f4a260d3df Simplify tests with tempfile::tempfile()
Where it simplifies the code, use tempfile() rather than TempDir to
create temporary files in tests.

BUG=None
TEST=./build_test

Change-Id: I5caff512a38a3b94556b0c72693e432503d6e679
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2360459
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
2020-08-27 00:39:03 +00:00
Andrew Walbran
02a17bb588 sys_util: Log to logd via liblog on Android.
The Linux syslog hack won't work, as Android doesn't use the same syslog
interface.

This introduces a Syslog trait to abstract over the syslog interface for
the target platform, which is implemented for Linux and Android.

BUG=b:158290206
TEST=cargo test && cargo build --target=aarch64-linux-android

Cq-Depend: chromium:2355628
Change-Id: Ib7a86b37a1526a3ba9f7a0fdb47f1b959caf7c7c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2302469
Tested-by: Andrew Walbran <qwandor@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Auto-Submit: Andrew Walbran <qwandor@google.com>
2020-08-18 06:20:00 +00:00
Daniel Verkamp
32f423723a gpu_renderer: remove redundant -> () from fns
Fixes clippy "unneeded unit return type" warnings.

Also apply rustfmt.

BUG=None
TEST=bin/clippy
TEST=bin/fmt --check

Change-Id: I139c22e1f867de45cc1a5f4f460f08699d1007a9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2349954
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-08-13 01:28:52 +00:00
Dylan Reid
1330619561 sys_util: Add ability to set the depth of a pipe
It will be useful to set the pipe depth in unit tests. The first use
will be to set the depth before filling the pipe so the pipe can be used
to test blocking writes.

Change-Id: Ia8c2b01f7f4b08bd59a0442c5e7bff02bf25c77c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2345376
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
2020-08-13 00:04:54 +00:00
Keiichi Watanabe
156cea0d43 sys_util: Fix all the default clippy warnings
Fix following warnings so that cargo clippy passes under sys_utils:
* new_without_default
* wrong_self_convention
* or_fun_call
* comparison_chain

Note that we still have warnings on other targets such as --tests.

BUG=chromium:1111728
TEST=cargo clippy passes under sys_util directory

Change-Id: I286c4e0386e44a9aba5e78594a6f30c1046ef7ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2331885
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-08-06 03:28:13 +00:00
Keiichi Watanabe
94e8d811a5 sys_util: Fix clippy:missing_safety_doc warnings
BUG=chromium:1111728
TEST=cargo clippy

Change-Id: Ia9727cbf8041d660d219f1df3e0775d6e18e1fe3
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2331884
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-08-06 03:28:12 +00:00
Keiichi Watanabe
bb14010332 sys_util: Apply clippy's autofix
Ran the following command under /sys_util/:
`rustup run nightly cargo clippy --fix -Z unstable-options`

BUG=chromium:1111728
TEST=build

Change-Id: I2ecb04f8dea6711c9f0f658185ecdb201ba1f659
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2331883
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-08-06 03:28:09 +00:00
Daniel Verkamp
f2ea888204 sys_util: use is_null rather than == null_mut
Fixes a clippy warning:

  Comparing with null is better expressed by the `.is_null()` method

BUG=None
TEST=bin/clippy

Change-Id: I04d310107763075bc59fdadf7db8bd3c6f54500b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2317890
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
2020-07-27 21:36:23 +00:00
Dylan Reid
ec058d6c46 vm_memory: A crate to hold vm-specific memory objects
Move GuestAddress and GuestMemory to a new crate for VM memory. This
will make separating sys_util and crosvm independent making it easier
to use sys_util functions outside of crosvm.

Change-Id: I12e14948ea85754dfa6267b3a3fb32b77ef6796e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2311251
Auto-Submit: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-07-24 21:42:34 +00:00
Lingfeng Yang
d6ac1ab3a7 kvm: allow registering memory from current address space
This is good for enabling non-exportable Vulkan host coherent memory,
along with anything else where an exportable OS object isn't supported.

This CL introduces:

1. ExternalMapping, which wraps an external library mapping using
function callbacks, for purposes of sharing device memory to the guest
in the case where the device memory is not compatible with the mmap interface.
This is common in Vulkan when VkDeviceMemory is host visible but not
external, or external but based on an opaque fd.

The lifetime of the library mapping is tied to the lifetime of the
ExternalMapping.

2. Usually, we would send such memory requests over a socket to the main
thread. However, since these new objects require more metadata than
other requests that are sent over the wire (because there's information
about inheritance and refcounts), we also plumb the "map_request" field,
which wraps a single ExternalMapping.

Note that this ExternalMapping will not work in the sandbox case. In the
sandbox case, we will then have to figure out how to serialize/deserialize
ExternalMapping requests over a socket.

BUG=b/146066070, b/153580313
TEST=compile and test

Change-Id: I3b099b308aec45a313a8278ed6274f9dec66c30b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2034029
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
2020-07-24 00:19:11 +00:00
Noah Gold
ee02cb6f36 [data_model] VolatileSlice use IoSlice.
Switches from using libc::iovec directly to using our own IoSliceMut.
This also renames IntoIovec to IntoIobuf, along with the associated
methods. Effectively this pushes conversion into iovec closer to
where it is used.

Test: FEATURES=test emerge-eve crosvm
Change-Id: I2f907aa321a43d751a82e8430c74ac08b95772f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2300842
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Noah Gold <nkgold@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-07-21 23:20:22 +00:00
Keiichi Watanabe
36b2a0df71 crosvm: Fix clippy::needless_doctest_main
We can omit main() function in documantation tests.
cf. https://doc.rust-lang.org/rustdoc/documentation-tests.html#pre-processing-examples

BUG=none
TEST=cargo test
TEST=bin/clippy

Change-Id: Iaad756ac32d106b2b2bcee44c9301244f647f8f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2301675
Tested-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
2020-07-21 13:18:10 +00:00
Zach Reizner
94d77df61c enable core scheduling with the chromeos feature
This change includes support for core scheduling in sys_util and the
usage of it within the vcpu threads.

Because the core scheduling prctl is not yet stabilized, this feature is
specific to Chrome OS for now. To indicate this, the new "chromeos"
feature has been added to the appropriate Cargo.toml files.

TEST=cargo build --features "chromeos plugin"
BUG=b:153989878

Change-Id: I8b6ce00b26c482739536b632f08e55264dc513a0
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2284184
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Zach Reizner <zachr@chromium.org>
Auto-Submit: Zach Reizner <zachr@chromium.org>
2020-07-21 02:19:51 +00:00
Daniel Verkamp
f4f8f86873 Fix new clippy warnings from Rust 1.45.0
Fix a few instances of and_then(Ok(...)) in syslog:
https://rust-lang.github.io/rust-clippy/master/index.html#bind_instead_of_map

And an unused (overwritten) variable assignment in the argument tests.

BUG=None
TEST=docker/wrapped_smoke_test.sh

Change-Id: Id59b1749fd7eba1fe7dfeb7938ecbbc9d0f9aa6a
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2303275
Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
2020-07-17 06:49:30 +00:00
Dylan Reid
bae84398e5 sys_util: Move set_errno to the only place it's used
Move the OS specific errno handling to the handle_eintr test code which
is the only place that uses it.

The function was already only compiled for test code and the ifdefs have
started to spread with the added android support. Moving the code to the
test that uses it makes it more obvious that it is test-only code.

TEST=cargo test passes

Change-Id: I9fea1bb30052c0edc41c9609e2698221daefa580
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2296828
Reviewed-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Andrew Walbran <qwandor@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
2020-07-17 00:50:43 +00:00
Andrew Walbran
67448289e5 Use platform abstraction around libc::__errno_location.
It has a different name on Android.

BUG=b:158290206
TEST=cargo test

Change-Id: I424d0ddf24c008a06570c709354c7c2dd9395738
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2294865
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
2020-07-16 13:34:19 +00:00
Andrew Walbran
e43468f066 sys_util: Add IoctlNr type, as different platforms use different types.
In particular, Android targets use c_int whereas normal Linux targets
use c_ulong. An ioctl number should always fit in 16 bits, so casting
from c_uint (as bindgen uses for the constants) to either other type
should be safe.

BUG=b:158290206
TEST=cargo test

Change-Id: I3d808ba9a5588a75c029a299e3609d97e328e3cc
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2288229
Tested-by: kokoro <noreply+kokoro@google.com>
Auto-Submit: Andrew Walbran <qwandor@google.com>
Commit-Queue: Andrew Walbran <qwandor@google.com>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
2020-07-15 19:10:23 +00:00
Colin Downs-Razouk
883dbe845b sys_util: eventfd read_timeout function
Added a function to read from an eventfd with a timeout. This will be
useful for some IrqChip tests. Uses ppoll to check if an EventFd is
ready for reading.

Bug: chromium:1077058
Test: added a timeout test + cargo test -p sys_util
Change-Id: I10283e39ee21305c3e433aded417a7eea465def9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2276321
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Colin Downs-Razouk <colindr@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-07-06 19:12:09 +00:00
Colin Downs-Razouk
52e0cbf40c devices: pit updates for hypervisor abstraction
Update the pit to have get/set functions so it can be used as part of an
irqchip. Some of this translation is imprecise because we use a rust
Instant for storing the timer start time, where the PitState expects it
in terms of the host's monotonic clock, but it shouldn't really matter
because getting/setting the pit is only exposed as a plugin feature.

This also required adding some convenience functions to the sys_util
Clock.

Bug: chromium:1077058
Test: cargo test -p devices
Change-Id: I761747cd03ed123a2d2e685e0679a3d025a165ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2265043
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Colin Downs-Razouk <colindr@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-07-06 19:12:08 +00:00
Colin Downs-Razouk
9fef2380da sys_util: EventFd using SafeDescriptor
Update EventFd to use SafeDescriptor and add equality check to
SafeDescriptor. Two safe descriptors are equal if their underlying
RawFds are equal or if they are duplicates of each other created by
calling try_clone, which uses dup.

This will be used by the KvmSplitIrqChip so it's unregister_irq_event
functionality can match KVM's, which will not unregister an irqfd if the
event fd does not match.

Bug: chromium:1077058
Test: added a test for equality of a cloned SafeDescriptor, and ran all
 other sys_util tests
Change-Id: I8c55a083b4e0d693c7192fa869a60e192829db5e
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2265042
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Colin Downs-Razouk <colindr@google.com>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
2020-07-06 03:18:56 +00:00
Steven Richman
7c359d617f hypervisor: add Vm user memory region functions
The separate Vm functions for MemoryMappings and MemoryMappingArenas
have been combined and now use a MappedRegion trait that the mappings
implement.  msync_memory_region replaces the get_mmap_arena function,
which is used by VmMsyncRequest.  Since Vm uses mutexes for cloning, it
can't return mem region references.

BUG=chromium:1077058
TEST=cargo test, cargo test -p sys_util, cargo test -p hypervisor

Change-Id: If257b16ee34d07820ae7ebdb9a3a598a41df013c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2202845
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Gurchetan Singh <gurchetansingh@chromium.org>
2020-06-10 17:08:55 +00:00
Chirantan Ekbote
e618bf7ec5 sys_util: Make ioctl number method a const fn
This allows us to define const variables that are the return value of
the method, which we can then use in match statements.

BUG=b:157189438
TEST=unit tests

Change-Id: I2475c59bfd43ec9ec149a6b688bf680fa2361a0b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2214962
Tested-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
2020-06-04 06:09:30 +00:00
Chirantan Ekbote
1a9f2a5454 sys_util: Refactor IntoIovec
The original stated purpose of this trait was to reduce memory
allocations but having the `into_iovec` method return a Vec kind of
defeats that purpose.

Refactor the trait so that it can either convert a T into an iovec or
convert a &[T] into a &[iovec].  Implement the trait for VolatileSlice,
IoSlice, and IoSliceMut and update all the callers.

BUG=none
TEST=unit tests

Cq-Depend: chromium:2210272
Change-Id: I9d0d617a23030d241d50411f4a5a16e7cba4bcee
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2208527
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Commit-Queue: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: Chirantan Ekbote <chirantan@chromium.org>
2020-05-28 07:14:58 +00:00