Mix MP3 and generatedSoundStream #1828
-
|
Did anyone find a way to mix a generatedSoundStream and a decoded mp3 file? Firstly, when using a output mixer together with the encoded mp3 the playback speed gets too fast. I2SStream i2s;
OutputMixer<int16_t> mixer(i2s, 2);
VolumeStream volumeSong(mixer);
VolumeStream volumeWhiteNoise(mixer);
WhiteNoiseGenerator<int16_t> whiteNoise;
GeneratedSoundStream<int16_t> whiteNoiseIn(whiteNoise);
EncodedAudioStream decoder(&volumeSong, new MP3DecoderHelix());
StreamCopy copierSong;
StreamCopy copierWhiteNoise(volumeWhiteNoise, whiteNoiseIn);
void setup() {
auto config = i2s.defaultConfig(TX_MODE);
whiteNoiseIn.begin(config);
whiteNoise.begin();
i2s.begin(config);
volumeSong.setVolume(0.3);
decoder.begin();
mixer.begin(4608);
currentSong = SD.open("/001.mp3");
copierSong.begin(decoder, currentSong);
}
void loop() {
copierWhiteNoise.copy();
copierSong.copy();
}Error: Is this even possible with the resources of an esp32? The end goal is to have have a analog radio like audio where i can fade out the white noise and fade in the mp3 file. |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 11 replies
-
|
You have 2 major problems to solve:
|
Beta Was this translation helpful? Give feedback.
-
|
Firstly thanks for your answer :) Using the input mixer would be preferable for me. I understand how to wire up the white noise generator to it but im struggling with connecting the encodedaudiostream. Thing i have tried: MemoryStream memStream(currentSong, currentSong.size()); // Doesnt match the constructor EncodedAudioStream dec(¤tSong, new MP3DecoderHelix());
i2s.begin(config);
dec.addNotifyAudioChange(i2s);
dec.begin();
mixer.add(dec);
mixer.begin(config);Can you point me into a direction or give me some keywords so i can figure this out? Thanks a lot :) |
Beta Was this translation helpful? Give feedback.
-
|
I don't quite understand your question: you can use both the whiteNoiseIn and decoder from your original sketch as components for your InputMixer. The Mixing Wiki describes how to do mixing on the input side and the Codec Wiki has a chapter Decoding on the Input Side which is just showing that you can read PCM data from a EncodedAudioStream |
Beta Was this translation helpful? Give feedback.
-
|
Original decoding example: #include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include "BabyElephantWalk60_mp3.h"
MemoryStream mp3(BabyElephantWalk60_mp3, BabyElephantWalk60_mp3_len);
EncodedAudioStream dec(&mp3, new MP3DecoderHelix());
I2SStream i2s;
StreamCopy copier(i2s, mp3);
void setup(){
Serial.begin(115200);
i2s.begin();
dec.addNotifyAudioChange(i2s);
dec.begin();
}
void loop(){
copier.copy();
}After thinking about it some more my confusion comes from this i think: How does the copier get the decoded data then? When wanting to change the MemoryStream to a File input with the following sketch i just get white noise. (Im guessing because im passing in non-decoded data and thats just what it sounds like) #include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include <SD.h>
File mp3File;
EncodedAudioStream dec(&mp3File, new MP3DecoderHelix());
I2SStream i2s;
StreamCopy copier(i2s, mp3File);
void setup() {
Serial.begin(115200);
if (!SD.begin()) {
Serial.println("SD card initialization failed!");
stop();
}
mp3File = SD.open("/90.0/001.mp3");
if (!mp3File) {
Serial.println("Failed to open MP3 file!");
return;
} else {
Serial.println("Opened the MP3 file");
}
dec.begin();
AudioInfo info = dec.audioInfo();
auto config = i2s.defaultConfig(TX_MODE);
config.sample_rate = info.sample_rate;
config.channels = info.channels;
i2s.begin(config);
delay(100);
dec.addNotifyAudioChange(i2s);
}
void loop() {
copier.copy();
} |
Beta Was this translation helpful? Give feedback.
-
|
It should be StreamCopy copier(i2s, dec); copy want to copy the decoded data to i2s |
Beta Was this translation helpful? Give feedback.
-
|
Alright. It seems like like the program crashes because the amount of data written changes (always) after the 3rd loop. Code: #include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include <SD.h>
File mp3File;
EncodedAudioStream dec(&mp3File, new MP3DecoderHelix);
I2SStream i2s;
StreamCopy copier(i2s, dec);
void setup() {
// LOGLEVEL_HELIX = LogLevelHelix::Debug;
AudioLogger::instance().begin(Serial, AudioLogger::Debug);
Serial.begin(115200);
if (!SD.begin()) {
Serial.println("SD card initialization failed!");
stop();
}
mp3File = SD.open("/90.0/001.mp3");
if (!mp3File) {
Serial.println("Failed to open MP3 file!");
stop();
return;
} else {
Serial.println("Opened the MP3 file");
}
auto config = i2s.defaultConfig(TX_MODE);
// config.sample_rate = 48000;
i2s.begin(config);
dec.addNotifyAudioChange(i2s);
dec.begin();
}
void loop() {
copier.copy();
}Error Log: |
Beta Was this translation helpful? Give feedback.
-
|
Not sure if this will help, did you try to decrease the copy size ? e.g. with StreamCopy copier(i2s, dec, 256); |
Beta Was this translation helpful? Give feedback.
-
|
I did now, and i tried some more values but all that achieved was just that it happened later in the program because it takes longer to reach that part in the mp3 file i guess. I have also checked the mp3 file using ffmpeg and like i said it has worked when using on the ouput side. |
Beta Was this translation helpful? Give feedback.
-
|
The EncodedAudioStream read is decoding the data into a queue and it seems that the queue was not big enough for the big decoded arrays created by the mp3 decoder. I committed a correction which adjusted the queue size to make it work with libhelix. If this is not working for you you can try to incread the buffer size: with a copy size of 1024 bytes 5 means 5 * 1024 bytes: |
Beta Was this translation helpful? Give feedback.
-
|
Yes this worked. Thank you for the fix :) FYI i had to increase the resultQueueFactor by 14 to get it working. |
Beta Was this translation helpful? Give feedback.
-
|
Was this with the StreamCopy default size or with a decreased value ? |
Beta Was this translation helpful? Give feedback.
-
|
Default size: #include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include <SD.h>
File mp3File;
EncodedAudioStream dec(&mp3File, new MP3DecoderHelix);
I2SStream i2s;
StreamCopy copier(i2s, dec);
void setup() {
// LOGLEVEL_HELIX = LogLevelHelix::Debug;
// AudioLogger::instance().begin(Serial, AudioLogger::Debug);
Serial.begin(115200);
if (!SD.begin()) {
Serial.println("SD card initialization failed!");
stop();
}
mp3File = SD.open("/95.0/001.mp3");
if (!mp3File) {
Serial.println("Failed to open MP3 file!");
stop();
return;
} else {
Serial.println("Opened the MP3 file");
}
auto config = i2s.defaultConfig(TX_MODE);
// config.sample_rate = 48000;
i2s.begin(config);
dec.addNotifyAudioChange(i2s);
dec.transformationReader().setResultQueueFactor(14);
dec.begin();
}
void loop() { copier.copy(); } |
Beta Was this translation helpful? Give feedback.
-
|
Interesting thing i have noticed: #include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include <SD.h>
File mp3File;
EncodedAudioStream dec(&mp3File, new MP3DecoderHelix);
I2SStream i2s;
VolumeStream volume(i2s);
StreamCopy copier(volume, dec);
void setup() {
// LOGLEVEL_HELIX = LogLevelHelix::Debug;
// AudioLogger::instance().begin(Serial, AudioLogger::Debug);
Serial.begin(115200);
if (!SD.begin()) {
Serial.println("SD card initialization failed!");
stop();
}
mp3File = SD.open("/95.0/001.mp3");
if (!mp3File) {
Serial.println("Failed to open MP3 file!");
stop();
return;
} else {
Serial.println("Opened the MP3 file");
}
volume.setVolume(0.1);
auto config = i2s.defaultConfig(TX_MODE);
// config.sample_rate = 48000;
i2s.begin(config);
dec.addNotifyAudioChange(i2s);
dec.transformationReader().setResultQueueFactor(14);
dec.begin();
}
void loop() {
copier.copy();
Serial.printf( "Current volume: %f\n", volume.volume()); // this prints correct as: Current volume: 0.100000
}This also does not work if you try to use the volumeStream on the input side: #include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include <SD.h>
File mp3File;
EncodedAudioStream dec(&mp3File, new MP3DecoderHelix);
I2SStream i2s;
VolumeStream volume(dec);
StreamCopy copier(i2s, volume);
void setup() {
// LOGLEVEL_HELIX = LogLevelHelix::Debug;
// AudioLogger::instance().begin(Serial, AudioLogger::Debug);
Serial.begin(115200);
if (!SD.begin()) {
Serial.println("SD card initialization failed!");
stop();
}
mp3File = SD.open("/95.0/001.mp3");
if (!mp3File) {
Serial.println("Failed to open MP3 file!");
stop();
return;
} else {
Serial.println("Opened the MP3 file");
}
volume.setVolume(0.1);
auto config = i2s.defaultConfig(TX_MODE);
// config.sample_rate = 48000;
i2s.begin(config);
dec.addNotifyAudioChange(i2s);
dec.transformationReader().setResultQueueFactor(14);
dec.begin();
}
void loop() {
copier.copy();
Serial.printf( "Current volume: %f\n", volume.volume()); // this prints correct as: Current volume: 0.100000
}Both examples compile and play the file but do not change the volume |
Beta Was this translation helpful? Give feedback.
-
|
you forgot to call volume,begin()! |
Beta Was this translation helpful? Give feedback.
-
|
it has been a long day i guess... |
Beta Was this translation helpful? Give feedback.
-
|
The original question was about mixing a mp3 stream and a generated sound. File audioFile;
I2SStream i2s;
InputMixer<int16_t> mixer;
WhiteNoiseGenerator<int16_t> whiteNoise;
GeneratedSoundStream<int16_t> whiteNoiseIn(whiteNoise);
EncodedAudioStream decoder(&audioFile, new MP3DecoderHelix());
VolumeStream volumeSong(decoder);
VolumeStream volumeWhiteNoise(whiteNoiseIn);
StreamCopy copier(i2s, mixer);
// Arduino Setup
void setup(void) {
Serial.begin(115200);
while (!Serial)
;
if (!SD.begin()) {
Serial.println("SD card initialization failed!");
stop();
}
audioFile = SD.open("/95.0/001.mp3");
if (!audioFile) {
Serial.println("Failed to open MP3 file!");
stop();
return;
} else {
Serial.println("Opened the MP3 file");
}
Serial.println("Card Mount succeeded");
auto config = i2s.defaultConfig(TX_MODE);
whiteNoiseIn.begin(config);
whiteNoise.begin();
i2s.begin(config);
volumeSong.setVolume(0.1);
volumeSong.begin();
volumeWhiteNoise.setVolume(0.1);
volumeWhiteNoise.begin();
decoder.setStream(currentSong);
decoder.transformationReader().setResultQueueFactor(14);
decoder.begin();
mixer.add(volumeSong);
mixer.add(volumeWhiteNoise);
mixer.begin(config);
copier.begin();
}
void loop() {
copier.copy();
} |
Beta Was this translation helpful? Give feedback.
You have 2 major problems to solve: