enter can fail if there are too many pending ops. This will return
EBUSY. The user will have to handle some completions before trying
again. Only count sqes as submitted by resetting self.added when enter
returns success.
The number of sqes must also be guarded if too many are added before the
kernel can update the head pointer.
TEST=added ebusy unit test
Change-Id: I0c20677f1332732aab09f025e110eca0c2a42500
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2296699
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
These will be needed by by block.
Note that support for uring fallocate was added in 5.6. The users will
have to handle falling back to synchronous calls if fallocate returns
EINVAL.
BUG=901139
TEST=added fallocate unit tests
Change-Id: I51d635adcf0bb4dd55c5bfe50719f2fde2b88e49
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2274996
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
It has a different name on Android.
BUG=b:158290206
TEST=cargo test
Change-Id: I424d0ddf24c008a06570c709354c7c2dd9395738
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2294865
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Andrew Walbran <qwandor@google.com>
Submission queue entries have whatever value is left from last time
stored. Clear them to zero when they are pulled out prior to being
filled. It's better to do this in a central location than making each
user responsible for setting all the fields.
Change-Id: I0436e24cd97fe4bbe808d7d0cc09fa81c1323d57
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2274995
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
The value is ignored if WAIT isn't indicated in the flags, but it's more
clear what is happening if left as zero as well.
Change-Id: Ib3539e37d1eaad6f5a68f6c028cdd186f12a8dd4
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2274994
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
uring.rs wraps the kernel io_uring interfaces that provide C-style raw
pointers, which requires casting a *u8 into other types with stricter
alignment requirements. All of these casts are based on the base mmap()
pointer (which is always page aligned) or offsets within an mmap
provided by the kernel, so we can assume they are safe to cast to the
relevant more-aligned types.
Fixes clippy warnings of the form:
casting from `*mut u8` to a more-strictly-aligned pointer (`*const
std::sync::atomic::AtomicU32`) (1 < 4 bytes)
BUG=None
TEST=bin/clippy
Change-Id: Ibef32bb8529d34ece79f089992ad8bd880c5030b
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2253068
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
clippy warns about the redundant 'static lifetime, which is implied for
const.
BUG=None
TEST=bin/clippy
Change-Id: I1d4dca90de359857ec947e12ae47858785d8867c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2253065
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Daniel Verkamp <dverkamp@chromium.org>
Commit-Queue: Daniel Verkamp <dverkamp@chromium.org>
A recent change to sys_util moved the definition of 'as_ptr' for an mmap
to a new trait, include that trait here.
Change-Id: Ib48113738fdace50e2dc7f72a5107dd95db867f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2241590
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
The pointer passed to the kernel for the iovecs must remain valid until
the operation completes. This is true even if the kernel copies the sqe
containing the pointer, that isn't a deep copy. If the iovecs are freed
before the kernel actually processes the op, it will fail with a bad
address error.
Move the responsibility for maintaining the list in memory from the
caller to io_uring itself. This mean io_uring must allocate. Taking
iterators for IoSlice/IoSliceMut, means that the caller doesn't need to
allocate as well if there isn't a Vec already allocated.
Change-Id: I63a009d8ab543c8bac4132809fb851536d4ad82c
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2227082
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
The kernel is allowed to not only limit the number of requested entries,
but also provide more than is requested. If a non-power-of-2 is
requested, it will be rounded up to the next power of 2.
Ensure that there are enough iovecs by allocating them based on the
number returned from io_uring_setup instead of the number of entries
requested.
TEST=cargo test read_parallel
Change-Id: If92e0a31858d1af53b99af8415002a4f26b48230
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2140915
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Let the user of the io_uring interface specify a slice of iovecs to read
from or write to when queueing an operation to the ring.
These ops can be used by block that has a Vec of iovecs from the
descriptor chain already.
Change-Id: Ia91e03e441cdae03e4fdba33bb601de006ef53ef
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2140914
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Switch from returning an i32 to a u32. This will make handling the
number easier for users, as they can assume it is >= 0.
Any value < 0 would not be returned as Ok(value) anyways as ret < 0 is
used for error conditions.
Change-Id: I609ce55d3c6be6e28f4d7aadf7148b2ac3b18878
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2140913
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Tested-by: Dylan Reid <dgreid@chromium.org>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
Provide a low-level interface to operating the new io_uring interface.
This is an unsafe interface with the basic operations of setting up the
ring, adding to it, removing from it, and polling it.
This will be followed by a safe interface layer on top of this code,
then by additions to the asynchronous executor that allow for
asynchronously doing multiple operations to files from one context.
Change-Id: I71f7ffb04ce8cb4da470deda9aee768ab95d3d98
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/2124009
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Zach Reizner <zachr@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Tested-by: kokoro <noreply+kokoro@google.com>
Commit-Queue: Dylan Reid <dgreid@chromium.org>
