Sample Management
Now that we know loading and playing of audio works in principle we can think a bit about making the process more robust and flexible. In this chapter some concepts such as classes, threads and thread-safe techniques are introduced. This chapter is based on this tutorial which I used to learn how to make loading sound files work in the plugin.
Before we get into rewriting the loading functions we'll start by doing a preliminary exercise on writing classes: the encapsulation of our processing function in a custom class. This class will be called Grain and over the next few chapters we will fill it with all the functionality we'll eventually need to do granular synthesis.
Classes are blueprints for objects and are made up of member variables (or attributes) and member functions (or methods). Our Grain class will encapsulate all the variables and functions that are necessary to play a single grain.
The Grain class will live in it's own header file so it's a little easier to read and reuse. New files are best added to the project with the Projucer:
You need to re-save the Projucer project for the file to appear in your IDE.
Now we need to add a include statement to the file "PluginProcessor.h":
#include "Grain.h"When implementing a class it is always a good idea to take a moment and think about what information should be stored in that class. For now it will just be one function which takes two buffers as arguments and writes a sample at a specific position (which is also passed as an argument) from one buffer into the other. We could also imagine the class holding it's own information about the position from which to read but for reasons that we will go into in the next chapter I implemented the plugin with a "global" position from which everything derives.
Starting out our Grain class will look like this:
class Grain
{
// [1]
public:
Grain(){};
~Grain(){};
// [2.1]
void process (AudioSampleBuffer& currentBlock, AudioSampleBuffer& fileBuffer, int numChannels, int blockNumSamples, int fileNumSamples, int time)
{
// [2.2]
for(int channel=0; channel<numChannels; ++channel){
float* channelData = currentBlock.getWritePointer(channel);
const float* fileData = fileBuffer.getReadPointer(channel%fileBuffer.getNumChannels());
// [2.3]
channelData[time%blockNumSamples] = fileData[time%fileNumSamples];
}
}
};- At the top of the class we see the access specifier
publicand the (empty) constructor and deconstructor. - This is the
processmember function. It is basically theprocessBlockfunction we wrote at the end of the last chapter with some changes: - The function now takes a few more arguments:
- The
AudioSampleBufferis now calledcurrentBlockto make the code more readable. - The second argument is a reference to a second buffer, here we will pass the buffer that holds the data in our sound file in. - We also have arguments for the number of channels and the number of samples in the block and in the loaded file. This function will be executed every sample (and later more than once for every sample). We don't expect these values to change much so we pass them in. - the last argument is an integer calledtimefrom which we will derive the position to read from. - We are now only iterating over the channels because the function will be called every sample from the
processBlockfunction in theAudioProcessor. - We are now using the function argument
timeinstead of thefilePositionvariable.
First we'll delete the filePosition variable and substitute it with new variable int time in the header of the PluginProcessor class and declare a variable Grain grain. Both of these variables are then defined in the PluginProcessor constructor:
GrrnlrrAudioProcessor::GrrnlrrAudioProcessor()
{
time = 0;
Grain grain = *new Grain();
}Our updated processBlock function looks like this:
void GrrnlrrAudioProcessor::processBlock (AudioSampleBuffer& buffer, MidiBuffer& midiMessages)
{
const int numSamplesInBlock = buffer.getNumSamples();
const int numSamplesInFile = fileBuffer.getNumSamples();
for (int i = getTotalNumInputChannels(); i < getTotalNumOutputChannels(); ++i)
buffer.clear (i, 0, buffer.getNumSamples());
// return from the function if there's nothing in our buffer
if(fileBuffer.getNumSamples() == 0) return;
for (int i=0; i<numSamplesInBlock; ++i){
grain.process(buffer, fileBuffer, buffer.getNumChannels(), numSamplesInBlock, numSamplesInFile, time); // [1]
++time; // [2]
}
}- We here access the member function
processof the grain object that we just wrote. - We increment the time by 1 every sample.
At this point one should once again compile and test if everything works!
Threads allow the simultaneous execution of two processes on the same CPU. Our program as it is implemented right now will be blocked for the duration of loading the sound file. This can be quite long depending on the size of the file and the speed of the system. In general we want to avoid this kind of behavior we want the user to still be able to interact with the GUI while loading the file for example. It is sensible to run the function that is doing the loading on a different background thread.
In JUCE the objects AudioProcessor and AudioProcessorEditor are already running on two seperate threads (called the audio and the messaging-thread) already. We can also implement a background thread on each of these objects by making the class inherit from the Thread class. For loading the buffer we will implement a background thread for the AudioProcessorEditor object. We do that by adding Thread to the list of classes AudioProcessorEditor inherits from:
class GrrnlrrAudioProcessorEditor : public AudioProcessorEditor,
public ThreadAs discussed before we sometimes have to implement virtual functions when we inherit from a class. To find out which functions we can take a look at the file with the class definition for Thread. All the definitions for the JUCE classes live in the 'modules' folder in the JUCE directory. This particular file can be found at modules/juce_core/library. A lot of IDEs also have a feature look up definitions for classes:
This is some of the information we find in the file 'thread.h':
/**
Encapsulates a thread.
Subclasses derive from Thread and implement the run() method, in which they
do their business. The thread can then be started with the startThread() method
and controlled with various other methods.
This class also contains some thread-related static methods, such
as sleep(), yield(), getCurrentThreadId() etc.
@see CriticalSection, WaitableEvent, Process, ThreadWithProgressWindow,
MessageManagerLock
*/We can see that we have to implement the run() method. To do that we first declare it in the header of the AudioProcessorEditor class:
void run() override;We also define the function:
void GrrnlrrAudioProcessorEditor::run()
{
}We also have to initialize the thread in the constructors initializer list, start it in the constructors body and stop it in the deconstructor:
GrrnlrrAudioProcessorEditor::GrrnlrrAudioProcessorEditor (GrrnlrrAudioProcessor& p)
: AudioProcessorEditor (&p), Thread("sample loading thread"), processor (p)
{
formatManager.registerBasicFormats();
startThread();
String path = "/Users/raffaelseyfried/dev/eigene/GRRNLRR/Resources/Piano_D11_High.wav";
loadSample(path);
setSize (400, 300);
}
GrrnlrrAudioProcessorEditor::~GrrnlrrAudioProcessorEditor()
{
stopThread(4000);
}Note what we give the thread a name when we initialize it. This is quite useful for debugging purposes. This is a good point to recompile and see if everything is still working.
What we want to implement now is an algorithm that checks in a predefined interval if a buffer needs to be loaded or deleted. To achieve this we declare two more functions in the AudioProcessorEditor class:
void checkForPathToOpen();
void checkForBuffersToFree();The loop we want to have in the end can be sketched out like this:
The run function first checks if the thread the function is running on is currently in the process of shutting down. If that's not the case checkForPathToOpen is called. checkForPathToOpen checks if there is a path to load a sample with. If there is then loadSample is called. Afterwards checkForBuffersToFree is called. This functions deletes any buffers that are not needed anymore. The run function looks like this:
void GrrnlrrAudioProcessorEditor::run()
{
while(! threadShouldExit()){
checkForPathToOpen();
checkForBuffersToFree();
wait(500);
}
}The exclamation mark (!) is a logical negation, changing true to false and false to true for boolean values. In this case it negates the bool threadShouldExit returns. So while the thread should not exit the run functions loops. The wait(int) causes the thread from which the function is called to wait for the time in millisecond specified by the first argument.
To implement the checkForPathToOpen function we first have to define another class. The class will get its own file named ReferenceCountedBuffer.h. We'll add the file to the project in the same way we added Grain.h.
This is the class:
#ifndef REFERENCECOUNTEDBUFFER_H_INCLUDED
#define REFERENCECOUNTEDBUFFER_H_INCLUDED
class ReferenceCountedBuffer : public ReferenceCountedObject
{
public:
typedef ReferenceCountedObjectPtr<ReferenceCountedBuffer> Ptr; // [1]
ReferenceCountedBuffer (const String& nameToUse,
int numChannels,
int numSamples) : name (nameToUse),
buffer (numChannels, numSamples) // [2]
{
DBG (
String ("Buffer named '") + name +
"' constructed. numChannels = " + String (numChannels) +
", numSamples = " + String (numSamples) );
}
~ReferenceCountedBuffer()
{
DBG (String ("Buffer named '") + name + "' destroyed");
}
AudioSampleBuffer* getAudioSampleBuffer() // [3
{
return &buffer;
}
private:
String name;
AudioSampleBuffer buffer;
JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (ReferenceCountedBuffer)
};
#endif // REFERENCECOUNTEDBUFFER_H_INCLUDEDReferenceCountedBuffer is a subclass of ReferenceCountedObject. ReferenceCountedObject is a base class that contains methods that manage the automatic deletion of objects once they are no longer needed. This employs the technique of 'reference counting'1 to test if an object is still in use.
- A class inheriting from
ReferenceCountedObjecthas to be made accessible with aReferenceCountedObjectPtrwith the type of the class. In our case this we would need to access the pointer like this:ReferenceCountedBuffer::ReferenceCountedObjectPtr<ReferenceCountedBuffer>. This is a quite unwieldy construction that we can simplify with atypedef. Atypedefallows us to give a type a different name. Here we give the typeReferenceCountedObjectPtr<ReferenceCountedBuffer>the namePtrand can access the type now withReferenceCountedBuffer::Ptr. - In the constructor we set defaults with the
:operator. We supply a name for the buffer for debugging and also supply the buffer itself. In the constructors function body we print the name of the buffer to the console. - We also define a method for accessing the buffer.
1: see also https://mortoray.com/2012/01/08/what-is-reference-counting/
Now we can implement checkForPathToOpen. First we declare a String variable named chosenPath in the header for the AudioProcessorEditor class:
String chosenPath;And implement the function:
void GrrnlrrAudioProcessorEditor::checkForPathToOpen()
{
String pathToOpen;
swapVariables(pathToOpen, chosenPath);
if(pathToOpen.isNotEmpty()){
std::cout << "We have a file!" << std::endl;
loadSample(pathToOpen);
}
}In the function we declare a variable String pathToOpen and swap the values of pathToOpen and chosenPath it with the JUCE function swap. pathToOpen open will be an empty string. With this we assure that chosenPath will always be empty after we execute checkForPathToOpen. This prevents that once we supply a path for chosenPath the program will try to load the sample multiple times.
Next we check if pathToOpen is still empty. If that is the case that means that we have no sample to load. If we have a valid path we first post a short message for debugging and then call the function loadSample with pathToOpen.
To test if everything is working we can temporarily define chosenPath in the constructor of the AudioProcessorEditor class:
[...]
startThread();
chosenPath = "/Users/raffaelseyfried/dev/eigene/GRRNLRR/Resources/Piano_D11_High.wav";
setSize (400, 300);
[...]If we compile and test the plugin now it should load and play the audio-file at the specified path.
Next we should rewrite the loadSample function with the ReferenceCountedBuffer. First we change the type of fileBuffer from AudioSampleBuffer to ReferenceCountedBuffer::Ptr:
ReferenceCountedBuffer::Ptr fileBuffer;Next we should comment out the whole function body of processBlock. This renders the code inert. We will worry about getting it to run again later incorporating the ReferenceCountedBuffer. We comment out by simply placing start (/*) and end comments (*/) around the function body like this:
void GrrnlrrAudioProcessor::processBlock (AudioSampleBuffer& buffer, MidiBuffer& midiMessages)
{
/*
const int numSamplesInBlock = buffer.getNumSamples();
const int numSamplesInFile = fileBuffer.getNumSamples();
for (int i = getTotalNumInputChannels(); i < getTotalNumOutputChannels(); ++i)
buffer.clear (i, 0, buffer.getNumSamples());
// return from the function if there's nothing in our buffer
if(fileBuffer.getNumSamples() == 0) return;
for (int i=0; i<numSamplesInBlock; ++i){
grain.process(buffer, fileBuffer, buffer.getNumChannels(), numSamplesInBlock, numSamplesInFile, time); // [1]
++time; // [2]
}
*/
}Afterwards we can rewrite the loadSample function:
void GrrnlrrAudioProcessorEditor::loadSample(String path)
{
const File file (path);
// we create the right kind of AudioFormatReader for our File
ScopedPointer<AudioFormatReader> reader(formatManager.createReaderFor(file));
ReferenceCountedBuffer::Ptr newBuffer = new ReferenceCountedBuffer(file.getFileName(),
reader->numChannels,
reader->lengthInSamples);
if(reader != nullptr){
// stream the contents of the Audio File into the Buffer
// args: AudioSampleBuffer*, startSample, endSample, readerStartSample, useLeftChan, useRightChan
// [1]
reader->read (newBuffer->getAudioSampleBuffer(), 0, reader->lengthInSamples, 0, true, true);
// [2]
std::cout << "Samples in Buffer: " << newBuffer->getAudioSampleBuffer()->getNumSamples() << std::endl;
// processor.fileBuffer = newBuffer;
}
}- Rather than supplying the
AudioSampleBufferdirectly to thereadmethod of theAudioFormatReaderwe here have to first make that buffer available through theReferenceCountedBuffer. - We also have to rewrite the text output accessing the
ReferenceCountedBuffer.
Here we should once again test if everything is still working. Note that the buffer shouldn't play at this point because we 'disabled' the processBlock function by commenting out the code. We should however see text output that the buffer has been loaded correctly.
Now we can also rewrite the processBlock function like this:
void GrrnlrrAudioProcessor::processBlock (AudioSampleBuffer& buffer, MidiBuffer& midiMessages)
{
// make a copy of the Pointer to the fileBuffer so we are completely seperated
// from the other thread
ReferenceCountedBuffer::Ptr retainedBuffer (fileBuffer);
// return from the function if there's no valid buffer
if(retainedBuffer == nullptr) return;
// finally get a AudioSampleBuffer we can use for processing
AudioSampleBuffer* currentBuffer = retainedBuffer->getAudioSampleBuffer();
const int numSamplesInBlock = buffer.getNumSamples();
const int numSamplesInFile = currentBuffer->getNumSamples();
for (int i = getTotalNumInputChannels(); i < getTotalNumOutputChannels(); ++i)
buffer.clear (i, 0, buffer.getNumSamples());
for (int i=0; i<numSamplesInBlock; ++i){
grain.process(buffer, *currentBuffer, buffer.getNumChannels(), numSamplesInBlock, numSamplesInFile, time);
++time; // increment time
}
}A few things are happening at once here. First we make a local (to the scope of this function) copy of the pointer to our buffer. Doing this the buffer inside processBlock is totally independent of the messaging-thread where a buffer could be deleted or added while the function is running. Next we see if retainedBuffer is pointing to a valid buffer. If not we return from the function. Then finally we can make the buffer accessible the buffer and proceed. In the following code all mentions of fileBuffer have been renamed to currentBuffer.
Large changes like this affecting multiple places in the program are always more error prone. We have to change many things before the program can compile again and we can test it. A strategy that has worked well for me in these situations was when something went wrong to first pin-point exactly where the failure was happening. I started by backtracking my steps and commenting out the code I had just written. When the program would compile and run again (with limited functionality) I would add the functions back in one by one until the error would start to happen again. Knowing where the error is happening is often (more than) half the battle here.