53ade023da
Fix the broken timestamp management in the virtio decoder device. Timestamps are specified by the guest's user-space as a struct timeval, which is effectively a (seconds, microseconds) pair. This pair is turned into a u64 with nanoseconds units by V4L2, and passed to the virtio device in that form. This is fine so far, save for the fact that libvda works with 32-bit timestamps. To accomodate that, we divided the original timestamp by 1_000_000_000, passed that truncated timestamp to the decoder backend as a u32, and when the backend gives us decoded frames, multiplied their timestamp by 1_000_000_000 again and passed that value as the frame timestamp to the guest. For some reason, we also used the timestamp as the key for the `NotifyEndOfBistreamBuffer` event, which forced us to have a mapping table between an input buffer timestamp and its corresponding resource ID. This in turn required that each input buffer has a unique timestamp, lest they collide in the mapping table. Anyway, the timestamp division by 1_000_000_000 means that any sub-second timestamp information was lost during the decoding process. This is not a problem with the Android V4L2 decoder which increases the timestamp by one second for each frame and does not use sub-second information, but clients that use *actual* timestamp information (like ffmpeg) get pretty confused by that and abandon decoding with an error. Fix this by passing the original 64-bit timestamp to decoder backends, along with the 32-bit ID of the input resource. This allows backends to directly send the NotifyEndOfBitstreamBuffer with the currect resource ID, removing the need for the mapping table and unique timestamps for input buffers. More importantly, backends that can work with 64-bit timestamps (all of them but libvda) can just pass the original timestamp to their decoder logic and return it as-is in the PictureReady event. Libvda still requires 32-bit timestamps, so we reproduce the truncation and mapping table behavior inside the libvda backend (I also tried sending the 32-bit input resource ID as a timestamp to VDA, but it was not happy with it). At least now that behavior is local to the VDA backend. Thanks to this change clients other than the Android C2 decoder can get proper timestamp information. One can argue that the code is also simpler (except for libvda which adds a simple mapping table that we were using before anyway). BUG=b:161774071 TEST=cargo test --features "video-decoder,ffmpeg" -p devices video TEST=`ffmpeg -codec:v vp8_v4l2m2m -i test-25fps.vp8 test-25fps-%d.png` in the guest completes successfully with the ffmpeg backend. TEST=tast arc.VideoDecodeAccel*_vm passes on hatch. Change-Id: Idb21b4c536acafbdf5458e88cdbc33c9376a405e Reviewed-on: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/3841480 Commit-Queue: Alexandre Courbot <acourbot@chromium.org> Tested-by: Alexandre Courbot <acourbot@chromium.org> Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@google.com> |
||
|---|---|---|
| .. | ||
| src | ||
| bindgen.sh | ||
| build.rs | ||
| Cargo.toml | ||
| libva-wrapper.h | ||
| README.md | ||
Libva Rust wrapper
Rust wrapper for libva. Provides safe libva abstractions for use within Rust code using its own bindgen-generated bindings.
This crate is used as part of the VirtIO Video VA-API backend. VA-API uses the GPU to perform the actual decoding/encoding of frames. In doing so, it frees the CPU for other uses and reduces power usage in the system. Hardware accelerated media workflows also tend to be faster than their software counterparts.
Note: This create requires the native libva library at link time. It also requires a VA-API driver to be installed on the system. The VA-API driver to use depends on the underlying hardware, e.g.: the implementation for Intel hardware is in intel-media-driver, whereas AMD hardware will depend on Mesa.
An easy way to see whether everything is in order is to run the vainfo utility. This is usually
packaged with libva-utils or as a standalone package in some distributions. vainfo will print
the VA-API version, the driver string, and a list of supported profiles and endpoints, i.e.:
vainfo: VA-API version: 1.13 (libva 2.13.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.2.2 ()
vainfo: Supported profile and entrypoints
VAProfileNone : VAEntrypointVideoProc
VAProfileNone : VAEntrypointStats
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
etc
For decoding, the desired profile must be supported under VAEntrypointVLD. For example, in order
to decode VP8 media, this line must be present in the output of vainfo:
VAProfileVP8Version0_3 : VAEntrypointVLD
Whereas to decode H264 Main profile media, this line must be present:
VAProfileH264Main : VAEntrypointVLD
For more information on VA-API and its usage within ChromeOS, see this guide.
For a brief introduction on how to use this crate, see the libva_utils_mpeg2vldemo test under
src/lib.rs.