I2S -> FFT stream is producing random numbers at most

[pikaro]

I tried to get I2S hooked up to FFT on an Arduino Nano ESP32-S3 all day yesterday, using a capture card very similar to the one described under "External ADC":

https://github.com/pschatzmann/arduino-audio-tools/wiki/External-ADC#i2s-adc-audio-i2s-capture-card-module

Only mine allows configuring some more options - I can switch between 16 and 24 bit samples and would have an option to switch between "LS", "RJ" and "I2S", which I suppose is the bit alignment. "I2S" is default, so presumably I2S_STD_FORMAT. The capture frequency can also be configured using a jumper.

Top view: https://m.media-amazon.com/images/I/71dZx+7XiyL._SL1500_.jpg
Bottom view: https://m.media-amazon.com/images/I/61C6QDsscKL._SL1500_.jpg

Unfortunately, there's no documentation available, but I tried all permutations I could think of / figure out:

• Slave mode
• 48k/98k/192k Master mode
• 16/24bit with 16/24/32 specified in code
• With or without specified mck pin
• With and without APLL
• Release / git versions of arduino-audio-tools
• arduino-cli and Platformio with core 2 / 3
• With and without windowing function, with and without buffer

For slave mode, I think the capture frequency is set by the master clock because it only lists fs values, which I think is a divisor for the master clock. Many combinations result in no I2S output being produced at all, some result in all-zero values, all-inf or all-ovf, and some produce numbers. However, those numbers seem unrelated to the audio input - I connected a synthesizer for testing, using both line-out and headphone jack.

The problem is that I'm not even sure which part of the system is not working. Soldering joints are straightforward and shouldn't be the issue, but even just the i2sStream.begin() function succeeds if the card is connected to entirely different pins. Is I2S actually working if I'm receiving data, any data at all? Or can this be a sync issue or something like that? Is it the way I interpret the output that's the issue (normalizing to bit length * FFT length for evaluation), or might the card just be broken?

In the end, what I want to do is perform FFT on a preamplified (mono) microphone input after applying simplistic AGC (which I take would be possible with arduino-audio-tools as well?), then produce FastLED animations from the resulting values. (Much more complex than the LEDOutput.h example would allow for.) That's also an open question: Do I actually have to continuously consume the I2S or FFT outputs, or can I just call the copier function and evaluate the results with e.g. 60 Hz? I won't be needing more resolution anyway. And if the I2S stream data must be consumed, can I run the copier in its own thread while limiting how frequently the FFT gives results to save on processing power?

This is one iteration of my code, for reference - maybe someone will notice something obvious, because I really have no idea what else to do here. Is there maybe any more granular, targeted way to trace things down inside the library? It's extremely abstract and requires lots of domain-specific knowledge it seems, and this is my first Arduino project so I'm a bit stuck.

#include "adc.h"

#include <Arduino.h>

#include <cmath>

#include "AudioLibs/AudioRealFFT.h"
#include "AudioLibs/FFTDisplay.h"
#include "AudioTools.h"
#include "common.h"

#define AUDIO_SAMPLE_RATE 48000
#define AUDIO_BITS_PER_SAMPLE 16

#define FFT_BANDS 8
#define FFT_LENGTH 512

#define USE_I2S
#define ADC_PIN_DIRECT GPIO_NUM_5
#define ADC_I2S_PIN_MCK GPIO_NUM_44
#define ADC_I2S_PIN_BCK GPIO_NUM_17
#define ADC_I2S_PIN_DATA GPIO_NUM_18
#define ADC_I2S_PIN_WS GPIO_NUM_19

#define FFT_MAX_MAGNITUDE (FFT_LENGTH * (1 << AUDIO_BITS_PER_SAMPLE))

using namespace std;

AudioInfo info(AUDIO_SAMPLE_RATE, 2, AUDIO_BITS_PER_SAMPLE);
I2SStream i2sStream;
AudioRealFFT fft;
FFTDisplay fftDis(fft);
StreamCopy copier(fft, i2sStream);

uint8_t fftBands[FFT_BANDS] = {
    5,                   // 156 Hz
    8,                   // 250 Hz
    16,                  // 500 Hz
    32,                  // 1 kHz
    64,                  // 2 kHz
    92,                  // 3 kHz
    128,                 // 4 kHz
    FFT_LENGTH / 2 - 1,  // 8 kHz
};

auto setup() -> bool {
    AudioLogger::instance().begin(Serial, AudioLogger::Warning);

    auto cfg = i2sStream.defaultConfig(RX_MODE);
    cfg.copyFrom(info);
    cfg.i2s_format = I2S_STD_FORMAT;
    cfg.channels = 2;
    cfg.is_master = false;
    cfg.use_apll = false;
    cfg.pin_ws = ADC_I2S_PIN_WS;
    cfg.pin_bck = ADC_I2S_PIN_BCK;
    cfg.pin_data = ADC_I2S_PIN_DATA;
    // cfg.pin_mck = ADC_I2S_PIN_MCK;
    if (!i2sStream.begin(cfg)) {
        slog("I2S initialization failed");
        return false;
    }
    slog("I2S initialization success");

    auto tcfg = fft.defaultConfig();
    tcfg.copyFrom(info);
    tcfg.length = FFT_LENGTH;
    tcfg.callback = &audio_tools::FFTDisplay::fftCallback;
    tcfg.window_function = new BufferedWindow(new Hamming());
    if (!fft.begin(tcfg)) {
        slog("FFT initialization failed");
        return false;
    }
    slog("FFT initialization success");

    fftDis.fft_group_bin = 16;
    fftDis.fft_start_bin = 0;
    fftDis.fft_max_magnitude = 40000;
    fftDis.begin();

    return true;
}

void loop() {
    for (;;) {
        copier.copy();

        for (int i = 0; i < FFT_BANDS; i++) {
            auto magnitude =
                static_cast<uint32_t>(round(fftDis.getMagnitude(i)));
            auto scaledMagnitude = static_cast<uint16_t>(round(
                255.0F * magnitude / static_cast<float>(FFT_MAX_MAGNITUDE)));

            Serial.printf("%10d (%3d)", magnitude, scaledMagnitude);
        }
        Serial.println();
    }
}

Device is jumpered to 48k, master, 16bit, I2S while running this. Sample output:

    189418 (  1)    135252 (  1)     67765 (  1)     83949 (  1)     69518 (  1)    146817 (  1)     99911 (  1)     62464 (  0)
    189418 (  1)    135252 (  1)     67765 (  1)     83949 (  1)     69518 (  1)    146817 (  1)     99911 (  1)     62464 (  0)
    141924 (  1)    114721 (  1)     82491 (  1)    167124 (  1)     98656 (  1)    102507 (  1)    117133 (  1)     84211 (  1)

[pschatzmann]

You are not supposed to display the result in the loop, but only when it is ready (as in this example)!

With such a high sample rate you might need to double check that your display is fast enough and not messing up the result because it is too slow....