Add a code path for better shared mode wasapi on windows #231

shangjiaxuan · 2020-02-15T05:40:09Z

Newer versions of windows exposes a event callback based shared mode, which has much smaller latency (up to less than 10ms) and min max frame counts and os mixing engine frame size can be queried through the new interface.

The added test in refresh device tests for the existence and sets a flag of availability in the device (I normally use c++, I don't know if there's bool type in pure c, if not, may need to change the bool to int), and the stream_do_open will open the new interface if detected earlier, and sets the smallest buffer size. The shared run code also added the path for this event based shared mode.

Since the queried size is in frame count, the path was changed to use frame count instead of reference time for better precision.

Tested on my device, before using this setup, the program continues to beep after a breakpoint is hit for a few seconds, corresponding to the default 4 second buffer size (a really large latency for anything, even multimedia playback is not allowed to have this lag). After using the new one, the time shortens to something acceptable (I cannot estimate with my ears).

Of course, setting the buffer size to something like 20ms in original code has similar effect, but that often ends up with lots of overhead. (It seems typically windows updates sound on a 10ms time frame, incompatible buffer sizes will impose overhead and dropouts.)

Add IAudioClient3 support

shangjiaxuan · 2020-02-15T06:03:05Z

It seems that on my device, windows updates only on a 10ms basis with 48000Hz floating point (480 frames, max and min are the same, increment also the same), and minimum buffer created has a size of 1056. It seems that the audio system on my device always asks for some padding for re-sampling and does some buffering at the same time with the new code path.

It may be that the buffer size is the total software latency on windows on my device, and buffers are only sent to engine after filled completely in shared mode (and the few seconds of sound after a breakpoint has been hit can be explained).

EDIT:
Running the latency.c will has correct timing of sound and console output time, but after breaking in (program execution paused), there will still be a few beeps before silence. The code seems to only take into account of the time between two writes of time. The audio buffer filled, however, is always much larger than the two writes, thus making the test invalid (only tests the performance of between-thread transfer of data.)

shangjiaxuan · 2020-02-15T07:05:23Z

It seems that there are also and issues pr not merged for this #174, will look further into it. Also #109 seems related.

wegylexy · 2020-07-30T13:36:43Z

Something has changed since Windows 10 v2004, does this patch solve those issues too?

wegylexy

Please remove the extra +.

wegylexy · 2020-07-30T21:52:25Z

src/wasapi.c

+static const IID IID_IAudioEndpointVolume = {
+    +    //MIDL_INTERFACE("5CDF2C82-841E-4546-9722-0CF74078229A")
+    +0x5cdf2c82, 0x841e, 0x4546, {0x97, 0x22, 0x0c, 0xf7, 0x40, 0x78, 0x22, 0x9a}
+};
+static const IID IID_IAudioEndpointVolumeCallback = {
+    +    //MIDL_INTERFACE("657804FA-D6AD-4496-8A60-352752AF4F89")
+    +0x657804fa, 0xd6ad, 0x4496, {0x8a, 0x60, 0x35, 0x27, 0x52, 0xaf, 0x4f, 0x89}
+};


You have extra + in front of these lines.

wegylexy · 2020-08-01T00:34:22Z

You mostly changed for outstream only. How about instream?

shangjiaxuan · 2020-08-03T03:39:40Z

This is mainly meant for better implementing a buffer size negotiation and callback based buffer submission. The main code path is no different from the original, only that the buffer can better accommodate the shared mode engine period, thus reducing the buffer size while not making audio dropouts.

I doubt if audio input will benefit from this as long as it is using a callback based implementation, since the available buffer size is determined by the os at wakeup time.

It is possible to change the os audio engine period to something much less than the default 10ms, but I did not have the hardware to test it (mine reports 480 to 480 samples working on native 48kHz float), thus currently only the current period is used.

Also when using smaller frames, like the raw mode on my device, there is a constant flow of dropouts. It seems that the thread scheduler cannot work on that much granularity, and must use the realtime work queue API for scheduling if such precision is needed.

The + was there when I was diffing, I'm sorry for that inconvenience. It was there probably when I was testing to compile as C code or CPP code, since the GUID const reference function prototype is different. The realtime work queue thing may need a lot more effort and change to implement.

shangjiaxuan · 2020-08-03T04:40:28Z

WIP

wegylexy · 2020-08-03T04:40:57Z

See also https://chromium.googlesource.com/chromium/src/media/+/refs/heads/master/audio/win/core_audio_util_win.cc for things like GetDevicePeriod, and handling of WAVEFORMATEX that is not WAVEFORMATEXTENSIBLE (e.g. PCM, IEEE_FLOAT).

Preparing for merge

Same as title.

shangjiaxuan · 2020-08-03T05:49:37Z

GetDevicePeriod only retrieves the device period, but in shared mode this does not map to the buffer submission size. In shared mode, the OS audio engine will gather buffer from all the apps and mix them (may or may not be in hardware), and then submit to the output. This shared mode buffer size is typically a few times larger than the output buffer size to enable mixing and resampling different streams to the hardware format. Shared mode cannot work on the buffer size retrieved from GetDevicePeriod without audio glitches. (And the smallest raw mode period typically cannot be achieved by the thread scheduler, with period that may be only 1-2ms, which is close to the thread cpu time frame.)

Thanks again for pointing to a working implementation, I'm adding a bit of non-WAVEFORMATEXTENSIBLE (8 bit and 16 bit pcm support, others may not work and should return invalid) in the WIP pull.

shangjiaxuan · 2020-08-03T05:50:46Z

The phnsDefaultDevicePeriod parameter specifies the default scheduling period for a shared-mode stream. The phnsMinimumDevicePeriod parameter specifies the minimum scheduling period for an exclusive-mode stream.

It seems that can be used when audio3 is not available, will put this into the WIP.

Also fix pausing logic that wwnt wrong some time ago.

shangjiaxuan · 2020-08-03T06:24:10Z

src/stdatomic.h

@@ -0,0 +1,163 @@
+#ifndef __MSVC_STDATOMIC_H


This atomic implementation is for msvc to build with c compiler, since windows sdk does not include a C atomic implementation (mingw should be configured to exclude this).

Code from github repo:
https://github.com/zenny-chen/simple-stdatomic-for-VS-Clang
under apache 2.0 license.
(maybe add a license note?)

wegylexy · 2020-08-03T22:20:11Z

src/config.h

+/*
+ * Copyright (c) 2015 Andrew Kelley
+ *
+ * This file is part of libsoundio, which is MIT licensed.
+ * See http://opensource.org/licenses/MIT
+ */
+
+#ifndef SOUNDIO_CONFIG_H
+#define SOUNDIO_CONFIG_H
+
+#define SOUNDIO_VERSION_MAJOR 2
+#define SOUNDIO_VERSION_MINOR 0
+#define SOUNDIO_VERSION_PATCH 0
+#define SOUNDIO_VERSION_STRING "2.0.0"
+
+/* #undef SOUNDIO_HAVE_JACK */
+/* #undef SOUNDIO_HAVE_PULSEAUDIO */
+/* #undef SOUNDIO_HAVE_ALSA */
+/* #undef SOUNDIO_HAVE_COREAUDIO */
+#define SOUNDIO_HAVE_WASAPI
+
+#endif


config.h is not meant to be checked in, is it?

wegylexy · 2020-08-03T22:44:56Z

src/wasapi.c

+    if (wave_format->Format.wFormatTag == WAVE_FORMAT_EXTENSIBLE) {
+        from_channel_mask_layout(wave_format->dwChannelMask, layout);
+    }
+    else {


Unreachable due to assert() above.

wegylexy · 2020-08-03T23:04:53Z

src/wasapi.c

+            dev_w_raw->iaudio3_available=true;
+            dev_w_shared->iaudio3_available = true;
+            dev_w_raw->iaudio3_available = false;
+            dev_w_shared->iaudio3_available = false;


true or false?

Remove code added to test shared mode without iaudioclient3 and the assertion added when the extensible problem is first known

shangjiaxuan · 2020-08-04T00:32:58Z

Fixed problems listed. They are mainly code for testing and initial waveformatex support. Sorry that I forgot to look at the diffs and made such mistake.

Also fixed pausing the original shared mode (It seems only stopping the stream, and then supplying buffers to it will somehow restart it on my PC (sdk 1903) currently, may be because of the smaller buffer size from GetPeroid exposed this problem or because of driver change (the original 4 second buffer may be much larger than the time from pause to unpause).)

wegylexy · 2020-08-04T00:49:59Z

It works on v2004. Will test on v1909 soon.

This is mainly a sanity check, since paused stream should not be calling callbacks

0x08088405 · 2021-06-16T09:51:58Z

Newer versions of windows exposes a event callback based shared mode,

@shangjiaxuan Sorry for bumping an old issue but is event mode exclusive to newer versions of IAudioClient? I know MSDN isn't great but does it mention this anywhere? Writing my own implementation based on libsoundio and that caught my eye.

shangjiaxuan added 6 commits February 14, 2020 15:03

Update .gitignore

643b262

Add audio client 2 & 3 iids

266ae7d

Add better shared code path

0a99925

Add IAudioClient3 support

Outstream on win10 now uses event based method for output

4384ab8

Merge remote-tracking branch 'upstream/master'

4b38f8e

Ready for merge

3af3024

shangjiaxuan mentioned this pull request Feb 15, 2020

Huge latency on Windows #221

Open

shangjiaxuan added 2 commits February 15, 2020 17:23

Fix broken exclusive mode

8cf7949

Fix broken exclusive mode

556d068

wegylexy suggested changes Jul 30, 2020

View reviewed changes

shangjiaxuan added 3 commits August 3, 2020 11:46

Cleanup

0f799f5

Clean up

8473722

merge

e7a2d15

shangjiaxuan closed this Aug 3, 2020

shangjiaxuan added 2 commits August 3, 2020 12:41

Add declaration for lower version of sdk

d83b92a

Preparing for merge

Add primitive support for non-extensible wave format

7013091

Same as title.

Add period negotiation with shared mode

9b9202b

Also fix pausing logic that wwnt wrong some time ago.

shangjiaxuan reopened this Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

View reviewed changes

Add license note for external stdatomic implementation

1406afe

wegylexy reviewed Aug 3, 2020

View reviewed changes

fix problems mentioned

13b1d70

Remove code added to test shared mode without iaudioclient3 and the assertion added when the extensible problem is first known

remove config from src dir

97d98b8

It seems raw mode does not respect the pause flag

dfa2c68

This is mainly a sanity check, since paused stream should not be calling callbacks

shangjiaxuan mentioned this pull request Aug 4, 2020

Unable to pause WASAPI output stream from callback #235

Open

shangjiaxuan mentioned this pull request Aug 20, 2020

因为不是主仓库fork，不能从其它fork给pull过来 MickeyMeowMeowHouse/libsoundio#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a code path for better shared mode wasapi on windows #231

Add a code path for better shared mode wasapi on windows #231

shangjiaxuan commented Feb 15, 2020

shangjiaxuan commented Feb 15, 2020 •

edited

Loading

shangjiaxuan commented Feb 15, 2020

wegylexy commented Jul 30, 2020

wegylexy left a comment

wegylexy Jul 30, 2020

shangjiaxuan Aug 3, 2020

wegylexy commented Aug 1, 2020 •

edited

Loading

shangjiaxuan commented Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

wegylexy commented Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

shangjiaxuan Aug 3, 2020

wegylexy Aug 3, 2020

wegylexy Aug 3, 2020

wegylexy Aug 3, 2020

shangjiaxuan commented Aug 4, 2020 •

edited

Loading

wegylexy commented Aug 4, 2020

0x08088405 commented Jun 16, 2021

Add a code path for better shared mode wasapi on windows #231

Are you sure you want to change the base?

Add a code path for better shared mode wasapi on windows #231

Conversation

shangjiaxuan commented Feb 15, 2020

shangjiaxuan commented Feb 15, 2020 • edited Loading

shangjiaxuan commented Feb 15, 2020

wegylexy commented Jul 30, 2020

wegylexy left a comment

Choose a reason for hiding this comment

wegylexy Jul 30, 2020

Choose a reason for hiding this comment

shangjiaxuan Aug 3, 2020

Choose a reason for hiding this comment

wegylexy commented Aug 1, 2020 • edited Loading

shangjiaxuan commented Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

wegylexy commented Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

shangjiaxuan commented Aug 3, 2020

shangjiaxuan Aug 3, 2020

Choose a reason for hiding this comment

wegylexy Aug 3, 2020

Choose a reason for hiding this comment

wegylexy Aug 3, 2020

Choose a reason for hiding this comment

wegylexy Aug 3, 2020

Choose a reason for hiding this comment

shangjiaxuan commented Aug 4, 2020 • edited Loading

wegylexy commented Aug 4, 2020

0x08088405 commented Jun 16, 2021

shangjiaxuan commented Feb 15, 2020 •

edited

Loading

wegylexy commented Aug 1, 2020 •

edited

Loading

shangjiaxuan commented Aug 4, 2020 •

edited

Loading