Trying to get ml_synth_sampler_example to work...

[KontinuumLab]

Any help appreciated!

I’m trying to get ml_synth_sampler_example to work on my setup:

• I'm on a Wemos Lolin32 Lite, and a PCM-5102 i2s audio output.
• My setup does not have PSRAM.
• I'm using Arduino 1.8.19, because newer versions seem to not work with littleFS.
• I'm using the most recent version of the ML_SynthTools code from github.

With the ml_epiano_example I was able to get the functionality that I need, after some experimenting on this exact setup, but with the sampler I'm unable to get any sound output at all.
I'm trying to get a test note to sound, using the "PappRohrSample.wav" wav file data provided. At the end of void setup() in the sampler example is this line: “Sampler_NoteOn(0, 69, 127);” which I assume is to play a test note, but I haven’t been able to find this actual function in the library or example.
I feel confident that once I get basic sampler output, I should be able to figure out the rest of my code myself, as was the case with the epiano example…

Below is a description of my process - maybe someone can easily spot my stupid mistake. But on the other hand, maybe I’m completely misunderstanding the issue. Is the current version of ML_SynthTools even able to use the sampler, or should I wait for a future version?

Anyway, here goes:

I had to comment out some things in the example.
First I get a multiple definition error for “void Status_LogMessage(const char*)”, which is both in the file “ml_status_inline.h” and in the file “ml_synth_sampler_example.ino”
After fixing that, I get a whole series of multiple definition errors, like:
“FS_OpenFile()”,
“FS_CloseFile()”,
“FS_UseTempFile()”,
“readBytes(unsigned char*, unsigned int)”,
“fileSeekTo(unsigned int)”,
“getCurrentOffset()”
And many others…
After selectively commenting out a lot of that stuff, I got the code to compile and upload to my chip. On startup I get the following output from the Serial monitor (sadly with no audio output):

Hallo
ml_sampler
I2S configured using following pins:
	BCLK,BCK: 26
	WCLK,LCK: 25
	DOUT: 22
	DIN: -1

Loading data
no sd card
LittleFS Mount Successful!
List files ...

PappRohrSample.wav   	 15334
Sampler Example
Phaser Init ok
rev: 15329, 15329
Reverb is ready!
Loading data
no sd card
LittleFS Mount Successful!
List files ...

PappRohrSample.wav   	 15334
setup done!

[marcel-licence]

Hi,

I've checked the current status of the project. Some updates were missing. I've moved the file access to the library to avoid copy and paste.

Did you run the following code, before calling the NoteOn?

        WavToSmpl_FileToSingleNote(FS_ID_LITTLEFS, "/PappRohrSample.wav", W2S_ALL_NOTES);
        Sampler_InstrumentDone();

The sample is very short.

In addition to that, did you call 'Sampler_SetSampleBuffer(storage, storageBytes);' with a local memory storage?

Best regards,
Marcel

[KontinuumLab]

Hi Marcel. Thank you so much for getting back to me!
I downloaded your new code, both for the library and example.

I have now included “wavToSmpl” and “InstrumentDone”, and it appears to be working. At least it’s not breaking anything…

Still some doubts about “SetSampleBuffer”
Originally it’s inside the PSRAM section of setup(), all of which I have commented out because I don’t have PSRAM.
So I’ll insert it separately, but…. What are sensible values for “storage” and “storagebytes”?
I gather “storagebytes” is the size of the sample, right? That’s around 16kb (15334 bytes), so do I just put in 16000?
And “storage”..? Is that an address? I really have no idea what to put here, or how to decide on a suitable value.

Also - a small side quest:
I’m still really confused by not being able to find any of the actual functions for the sampler. In “ml_sampler.h” there are a bunch of function prototypes, like: “Sampler_SetSampleBuffer”, “Sampler_NoteOn” and a long etc., but in all of the library and the sampler example I’m unable to find the actual functions for any of these. Is this just me being stupid again, or is something actually missing…?

Thanks!
I can’t wait to get this up and running. So many instruments to build :)

[marcel-licence]

SetSampleBuffer is required to keep some space for a sample or multiple samples. In case you are loading just one sample the size of the sample would be sufficient like 16k.

You could code like this:

static uint8_t storage[16 * 1024];
uint32_t storagebytes = sizeof(storage);
SetSampleBuffer(storage, storagebytes);

storage shall be used in global scope.

The sampler is currently precompiled to avoid any side effects and compiling issues. I plan to keep the sampler open source. There are some modules which will be linkable only.

[KontinuumLab]

Thanks Marcel.
Took me a couple of weeks to get back to this (work has been crazy) but now it's working.
I was getting messed up sound out of the included sample, like a weird bubbly sound. That turned out to be something within the reverb and phaser effects. After commenting out everything to do with those (both in loop and setup) the sound is now working great and sounding clean. Just like with the epiano example I'm able to hook up some external capacitive sensors and play a scale with the sample. Looking good!

I hesitate to mark this topic as closed, because I will now transition to trying to plug in some of my own sf2 and wav samples, and I might have some more questions.

Anyway, your help is much appreciated Marcel!

[marcel-licence]

Thank you for your feedback. I am happy to hear that

[KontinuumLab]

Hi again Marcel. I'm back with more noob questions, sorry...
I managed to get my own .wav samples to work using your guide lines from above. Thanks!
I'm now trying to get sf2 samples to work, but have so far not had any luck.

I swapped out:
WavToSmpl_FileToSingleNote(FS_ID_LITTLEFS, "/PappRohrSample.wav", W2S_ALL_NOTES);
trying both of these options:

SF2ToSmpl_LoadCompleteSoundFont(FS_ID_LITTLEFS, "/Vibe_YI.sf2");
SF2ToSmpl_LoadAllSamplesFromSF(FS_ID_LITTLEFS, "/Vibe_YI.sf2");

I adjusted the sample buffer size to fit. I also included wav_to_sampler.h of course
The "Vibe" sample was extracted from Yoshi's Island using Polyphone
Nothing else was changed from my earlier code (which worked for .wav samples)

While the code does compile, I'm not getting any sound output.
Any ideas what I might be missing?

Thanks!!

[marcel-licence]

Hi,
using SF2ToSmpl_LoadCompleteSoundFont should work.
When using Polyphone, did you keep the Samples + Instrument + Preset required for the sound you want to keep?
Loading a complete soundfont goes through the presets list.

SF2ToSmpl_LoadAllSamplesFromSF would load the samples only. That should always work. Did you get any output in the serial monitor when loading the sound?
It may be set to another channel.

Best regards,
Marcel

[KontinuumLab]

Thank you Marcel.
I'm rather ashamed to admit that the error was with my own stupidity. Working on this intermittently between other projects has me forgetting some very basic things. Specifically that, when using littleFS you have to upload the data (sample) separately from the sketch, using the "ESP32 Sketch Data Upload" from the Arduino Tools menu.
So the sample is now working just like the .wav sample which is awesome - Sorry for wasting your time with this.

So I'm almost ready to start making instruments and super excited about that, but I do have one last question to pester you with. The maximum sample size that I'm able to fit in memory using your library on the Lolin32 lite boards that I use, is something like 28KB.

When trying to fit bigger samples, I get the following errors:

....... .ino.elf section `.dram0.bss' will not fit in region `dram0_0_seg'
....... xtensa-esp32-elf/bin/ld.exe: DRAM segment data does not fit.
....... xtensa-esp32-elf/bin/ld.exe: region `dram0_0_seg' overflowed by 102872 bytes

Like I mentioned, these boards don't have PSRAM.
When testing the "Arduino-STK" library for this same project, I was able to fit at least 2.5MB of samples in memory, but of course that was in the form of arrays in the sketch itself. To make this work, I edited the boards.txt file to let me use the full memory.
Now, using your library, the sketch of course doesn't fill up the 4MB of memory on the board by a long shot. Is there any way that I can open up some of all that left over memory for samples using your library?

If not, then I'll have to change to a Wrover-based board (which have PSRAM). They're a bit more expensive, but it seems worth it, given that your library is pretty much ideal for my project in every other way.

Thanks again!

[marcel-licence]

I am happy to hear that it works.
~512k is very small for the samples. There is also some memory organization, a part can be used static and another part can be accessed using malloc. In addition to that if effects are used you will loose more memory especially when using the delay effect.
I would highly recommend using PSRAM for samples. At the moment RAM is used to allow dynamic manipulation of the samples.
I could imagine to create an update which plays samples directly from flash (read only).
To keep the price low using the flash only would be the best solution. When I find some time I can check if that would be possible to integrate.

[KontinuumLab]

Yeah, I'm pretty excited about making some cool instruments using ML-SynthTools. I'll make sure to share my progress here.
Not sure what you mean about the 512k...? I'm only able to fit about 28k right now. But there is at least 3MB free on the board. 512k for samples would honestly be great. I feel like that would be sufficient for most of my projects, as I don't really plan on using effects like delay etc internally in the instrument. How would you go about having 512k available without PSRAM?
Anyway, I just ordered a Wrover board (ESP32 DEV-C), and will be doing some tests with that. Turns out it's only a couple of euros more expensive, so not a big deal.

Thank you so much for your work on this library, and your patience with my questions!