Stream audio from ESP32 over WiFi

[biomurph]

Hi,
This repository is amazing! Thank you all for all of the great work you are doing.

My question seems simple, but I am finding it hard to arrive at a good plan of attack.

I have an ESP32 and a MEMs microphone (analog? digital?) and I want to sample audio from the mic and send it over WiFi 'in real time' to any connected client. I would imagine that I would need at least 8Khz sample rate. I am running mono audio. I also have other data that I want to send over WiFi, but I don't need 8Khz, more like 256Hz (think accelerometer data). Right now, I am agnostic regarding the mic (analog? digital?), but I probably have to test both in order to find the lowest noise setup for my application.

Where is the best place to start experimenting with in the audio tools that are here? It is a giant sprawling mass of examples, and I mean that in a good way!

My initial thoughts are to have the ESP32 stand up a bare bones server on my WLAN so that I can connect to it via my laptop and receive the audio stream. I don't need to actually hear the audio in the laptop browser, the audio will go through DSP that will output derived info from the stream (think FFT power levels at specific frequencies, for example). That's why I am agnostic about whether the mic input is analog or digital. I just need to find the best way that allows enough bandwidth for me to send the other sensor data alongside the audio.

Hope to hear from you soon.
Thanks!

[pschatzmann]

A good way to start is by reading the Wiki and follow the advice given in 'It's not working'; try to learn and test each processing step separately before you put things together.

The audio quality using digtal mems microphones is usually better then the analog ones: Info can be found in the Hardware section.

I suggest that you have a look at the communication wiiki and examples.

I have never tried to send information over different channels, but I think you can try to use different ports and use separate tasks.
Another approach would be to use a BinaryContainer and send the accelerometer data as metadata along with the audio

[biomurph]

Hi,

I have been working off of the streams-generator-webserver_wav.ino example to get started, and I've been able to both start the WiFi connection in my own code and put the server.copy() function into it's own task, with the idea of multithreading the different data modalities that I want to use, as you mentioned in your answer. That works well, and now I want to add other features, for example.

• I would like to be able to turn on and off the audio. I think I get how to do that with the suspend() and resume() methods in my loop(). Is that the best way?
• I also want to use a WebSocket to send the accelerometer data. It looks like you are using an HTTP GET request from the client to start the connection. Could I use my WebSocket with your library? Or is there a better way?

Thanks for all these great tools!

[biomurph]

Thanks! This makes getting started less of a lift.
I think that I will target putting the copy to stream in a separate task, and update the slower sampled data in the loop.