crosvm/common/audio_streams/README.md

# Audio Server and Stream interfaces

The `audio_streams` crate provides a basic interface for playing audio. This will be used to enable
playback to various audio subsystems such as Alsa and cras. To start, an empty playback example
`NoopStreamSource` is provided.
audio_streams: Copy dir from adhd with commit history In adhd repository: git log --pretty=email --patch-with-stat --reverse -- \ audio_streams/ \| \ sed '/Change-Id/d' \| sed 's/Reviewed-on/> Reviewed-on/g' \| \ sed '/Tested-by/d' \| sed '/Commit-Queue/d' \| sed '/Signed-off-by/d' \| \ sed '/Commit-Read/d' \| sed '/Cq-Depend/d' \| sed '/Auto-Submit/d' \| \ sed 's/BUG/> BUG/g' \| \ sed 's/TEST/> TEST/g' \| \ sed 's/Reviewed-by/> Reviewed-by/g' > ~/cras-to-crosvm.patch In crosvm repository: - apply patch by git am --committer-date-is-author-date < ~/cras-to-crosvm.patch - squash and amend commits by git rebase --interactive Add the `audio_streams` crate The `audio_streams` crate provides a basic interface for playing audio. This will be used to enable playback to various audio subsystems such as Alsa and cras. To start, an empty playback example `DummyStreamSource` is provided. > BUG=781398 > TEST=Included Dummy Stream tests and boot crosvm with ac97 using the DummyStream for output. > Reviewed-on: https://chromium-review.googlesource.com/1402264 > Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_streams: Servers can return a list of open FDs To allow implementers of the StreamSource trait to be put in Linux jails, add an optional function that returns a list of file descriptors that need to be kept open when entering a jail. > Reviewed-on: https://chromium-review.googlesource.com/1416052 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_streams: Add audio capture interface Add `CaptureBufferStream` which provides interface for audio capture. This will be used to enable capture to various audio subsystems such as Alsa and cras. A null capture example `DummyCaptureStream` is provided. > BUG=chromium:932268 > TEST=Unit tests > TEST=Apply full patch set and run $ cras_test capture -f /tmp/res.raw to test capture functionality > Reviewed-on: https://chromium-review.googlesource.com/1530354 > Reviewed-by: Dylan Reid <dgreid@chromium.org> audio_streams: add direct read and write functions for audio buffers These functions are needed so that clients of audio_streams can make use of the buffer's internal slices directly. Before, the only way to access internal buffers was indirectly via the client passing in a slice of its own, which would require an extra copy when the client is using volatile memory sources which can not be safely turned into slices. > BUG=chromium:938767 > TEST=build crosvm with "eliminate mut from non-mut references" > Reviewed-on: https://chromium-review.googlesource.com/1639621 > Reviewed-by: Dylan Reid <dgreid@chromium.org> audio_streams: Updated trait object syntax Trait objects without `dyn` generate warnings with newer compiler versions. > Reviewed-on: https://chromium-review.googlesource.com/1714273 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> Add blueprint files for crosvm dependencies. Test: m libaudio_streams Test: m libcras_sys Test: m liblibcras Bug: 140201589 > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/1789920 > Reviewed-by: Dylan Reid <dgreid@chromium.org> > Reviewed-by: Ivan Lozano <ivanlozano@google.com> CRAS: libcras: Support more output formats Add support for U8, S24_LE, and S32_LE audio to libcras and cras_tests. 24bit samples are stored in 4 bytes of space, not 3. > BUG=chromium:1010667 > TEST=sox -n -r 48000 -c 2 -b 8 -e unsigned-integer output.raw synth 3 sine 100 && cras_tests playback -f U8 output.raw > TEST=sox -n -r 48000 -c 2 -b 32 output.raw synth 3 sine 100 && cras_tests playback -f S32_LE output.raw > TEST=sox -n -r 48000 -c 2 -b 24 output.raw synth 3 sine 100 && sox -t s24 -r 48000 -c 2 output.raw -t s32 output_24.raw vol 0.00390625 && cras_tests playback -f S24_LE output_24.raw > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/1856646 > Reviewed-by: Fletcher Woodruff <fletcherw@chromium.org> CRAS: cras_tests: Restructure arguments In preparation for adding control functionality to cras_tests, restructure Command by making AudioOptions a parameter of the Command enum, e.g Command::Capture(AudioOptions). This way, a control command can be later added as Command::Control(ControlOptions). > BUG=chromium:1008990 > TEST=cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/1865832 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> CRAS: audio_streams: Limits length in copy_cb Limits the size of index for accessing internal buffer in PlaybackBuffer and CaptureBuffer to prevent index-out-of-range panic. > BUG=b:146471328 > TEST=Build and test audio capture with stresstestapp > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/1980593 > Reviewed-by: Yu-Hsuan Hsu <yuhsuan@chromium.org> CRAS: cras_tests: add WAVE playback and capture Create structs WavSource and WavSink which adapt hound::WavReader and hound::WavWriter to implement the Read and Write traits, and use these to implement WAVE playback and capture. Dynamically select which source type to use based on the input/output file name. The audio is limited to integer sample formats, but supports all bit depths that CRAS supports. S24_LE samples are upsampled to S32_LE due to a quirk of the encoder used. > BUG=chromium:1008990 > TEST="sox -n -r 48000 -b 16 output.wav synth 5 sine 330 && cras_tests playback output.wav", check that sine wave is played accurately. > TEST="cras_tests capture output.wav && aplay output.wav", check that data is recorded properly. > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/1832584 > Reviewed-by: Dylan Reid <dgreid@chromium.org> CRAS: move shm_streams into audio_streams Move the shm_streams trait into the audio_streams crate so that systems that do not use CRAS can create audio backends implementing the shm_streams interface. Remove cras_types as all code in it has moved to cras-sys or audio_streams. Add a StreamEffect struct to audio_streams containing one effect, echo cancellation, as well as a conversion to CRAS_STREAM_EFFECT in cras-sys. > BUG=chromium:968724 > TEST=cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2040904 > Reviewed-by: Dylan Reid <dgreid@chromium.org> > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> CRAS: Add .gitignore Add .gitignore for audio_streams, cras-sys and libcras to ignore Cargo.lock as these are rust libraries not end products. Doc: https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html > BUG=None > TEST=cargo build > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2100454 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> > Reviewed-by: Dylan Reid <dgreid@chromium.org> CRAS: Implement FromStr and Default for StreamEffect 1. Implement FromStr and Default traits for StreamEffect. > BUG=b:149174845 > TEST=cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2053654 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> CRAS: Change new_stream trait of ShmStreamSource 1. Allow users to create streams with multiple effects by a slice of StreamEffect in ShmStreamSource::new_stream. 2. Implement FromIterator<&StreamEffect> for CRAS_STREAM_EFFECT. 3. Update CrasClient new_stream API. > BUG=b:149174845 > TEST=cargo build > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2095644 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> Allows audio_streams errors to be passed across threads Instead of returning Box<dyn Error>, we returns Error that impls Send + Sync, so that these errors can be passed across threads. It's needed for collecting the errors when we create streams in child threads. > BUG=b:149437381 > TEST=emerge-{BOARD} cras_tests > TEST=cargo test in audio_streams > TEST=cargo test in libcras > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2215772 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> Remove explicit lifetimes that can be elided Fixes crosvm `cargo clippy` errors: error: explicit lifetimes given in parameter types where they could be elided (or replaced with `'_` if needed by type declaration) > BUG=None > TEST=crosvm bin/clippy > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2225151 > Reviewed-by: Dylan Reid <dgreid@chromium.org> audio_streams: add num_channels in ShmStream trait Add num_channels() to allow to get the channel count of a stream. > BUG=b:157433024 > TEST=cargo build > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2259312 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_streams: Change timer start time in null_consumption_rate ``` let start = Instant::now(); ``` should be moved to earlier than ``` let mut stream_source = NullShmStreamSource::new(); ``` > BUG=chromium:1106557 > TEST=cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2303415 > Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org> audio_stream: Suppress cargo clippy warnings Suppress `cargo::type_complexity` and `clippy::too_many_arguments` warnings which are acceptable. Also, fixed a trivial clippy warnings. > BUG=chromium:1112251 > TEST=`cargo clippy` passes > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2331964 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_streams: change frame_rate to u32 in ShmStream and ShmStreamSource 1. Change frame_rate from usize to u32 for * StreamSource::new_playback_stream * StreamSource::new_capture_stream * ShmStreamSource::new_stream * DummyStream::new * DummyCaptureStream::new * NullShmStream::new * MockShmStream::new * CrasClient::create_stream * CrasClient::new_pinned_playback_stream * CrasClient::new_pinned_capture_stream * CrasStream::try_new * cras_audio_format_packed::new * WavSource * WavSink::try_new The frame_rate may supports up to 96000 khz and therefore the correct data type should be u32. 2. Add frame_rate() to ShmStream trait allow to get the sample rate of a stream. 3. Add sample_rate to * CrasShmStream * CrasShmStream::try_new 4. Fix some clippy issues in cras_tests > BUG=b:168082107 > TEST=cargo build && cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2434114 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> CRAS: fix clippy lint in audio_streams Pass SampleFormat by value to sample_bytes() function, since it implements Copy and is small. > BUG=None > TEST=None > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2541844 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_stream: Replace the use of Dummy* to Noop* Ableist language like dummy should be removed. Adding a NoopStreamControl that is a duplication of DummyStreamControl for future migration so we don't break crosvm's dependency. Will have a follow-up CL to remove DummyStreamControl once all existing dependencies are removed. > BUG=b:167649426 > TEST=./build_packages --autosetgov --board=hatch adhd audio_streams \ libcras crosvm > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2547616 > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_streams: Cache elapsed Duration in unit tests Cache start_time.elapsed() with `elapsed` to prevent "overflow when subtracting duration" error. > BUG=chromium:1161663 > TEST=`cargo test` > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2612305 > Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org> cras: Remove Dummy* for inclusive language Ableist language like dummy should be removed. NoopStreamControl has been added to replace and deprecate DummyStreamControl. Remove DummyStreamControl as all existing dependencies are removed. > BUG=b:167649426 > TEST=./build_packages --autosetgov --board=hatch adhd audio_streams \ libcras crosvm && src/platform/dev/contrib/search_blocked_words.sh > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2739020 > Reviewed-by: Yu-Hsuan Hsu <yuhsuan@chromium.org> resolve rust clippy 1.50 issues The issue was with unnecessary lifetimes and nested match statements that resolved quite cleanly in this change. > BUG=b:179277332 > TEST=cargo clippy with rustc 1.50+ > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2757729 > Reviewed-by: Dylan Reid <dgreid@chromium.org> audio_streams: Add next_[playback/capture]_buffer_async with async sleep. > BUG=b:179757101 > TEST=cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/2960197 > Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> > Reviewed-by: Chih-Yang Hsia <paulhsia@chromium.org> audio_streams: Add executor arg to next_[playback/capture]_buffer_async cras stream will need an executor to initialize its AsyncAudioSocket > BUG=b:179757101 > TEST=cargo test > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/adhd/+/3005176 > Reviewed-by: Chirantan Ekbote <chirantan@chromium.org> BUG=b:191511078 TEST=cargo build Change-Id: I2832583b29722deee2ece210b2e7e821768c3fd0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3045578 Auto-Submit: Chih-Yang Hsia <paulhsia@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> Reviewed-by: Dennis Kempin <denniskempin@google.com> 2019-01-09 04:15:03 +00:00			`# Audio Server and Stream interfaces`

docs: Use mdformat to format markdown files The tool is added to ./tools/fmt which will be called during ./tools/presubmit as well as by Kokoro during testing. This requires a new dev container version to bundle the mdformat tool in the container. Note: mdformat does not have any IDE integrations, but the prettier plugin with prose-wrap enabled and a line length of 100 provides almost identical results. BUG=None TEST=./tools/fmt --check Change-Id: I0ae5659eff8555df7c85c70e62095a1e116b98da Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/3416098 Reviewed-by: Keiichi Watanabe <keiichiw@chromium.org> Reviewed-by: Alexandre Courbot <acourbot@chromium.org> Tested-by: kokoro <noreply+kokoro@google.com> Commit-Queue: Dennis Kempin <denniskempin@google.com> 2022-01-25 23:14:43 +00:00			The `audio_streams` crate provides a basic interface for playing audio. This will be used to enable
			`playback to various audio subsystems such as Alsa and cras. To start, an empty playback example`
			`NoopStreamSource` is provided.