This project presents an interactive, low-cost, binaural, virtual acoustics embedded system through headphones. It utilizes an embedded computer with low-latency audio and a low-latency head-tracker. For binaural real-time synthesis a discrete circular convolution algorithm is applied. It caculates the sequence of a head-related transfer function (HRTF), with a backwards shifted sequence of a ring-buffer of the same size, that stores the audio input signal, with a multiply–accumulate (MAC) function. Because of the short filter lengths convolution in the time domain is suffcient. The code is written in C for Pure Data and is able to handle SOFA (Spatially Oriented Format for Acoustics) files. This work can serve as a tutorial on how to program a Pure Data external. The system resources are sufficient enough to handle SOFA databases up to 1550 positions and a filter length of 128 taps for three virtual loudspeakers. A filter length of 128 taps is considered as accurate representation of HRTFs. With this configuration the overall latency of the system is 41 ms, which is low enough to be adequate for most virtual audio applications. Interpolation of HRTFs between positions has not been considered in this project. To process larger SOFA databases with more positions for example with offline interpolated HRTFs, either the number of virtual loudspeakers has to be reduced or the processed block-size has to be increased, whereby the latter extends the latency of the system.
- SOFAlizer~ is based on the Pure Data external earplug~. https://puredata.info/downloads/earplug/releases/0.2
- Sofalizer~ utilizes the library libmysofa to handle SOFA files. https://github.com/hoene/libmysofa
- bela_SOFAlizer is based on the project BelaOnUrHead. https://github.com/theleadingzero/belaonurhead
Bela - A Beagle Bone Black with Xenomai real-time kernel extensions and with a custom expansion board
for synchronous ultra low-latency audio and sensor processing at audio rate. As audio and sensor data
is processed simultaneously, no jitter is introduced between them.
Further it features two "Programmable Real-time Units " (PRUs), that are 200 MHz microcontrollers
on the same chip as the CPU, with access to the same memory and peripherals as the CPU.
A custom audio codec and sensor ADC/DAC driver utilizes the BeagleBone PRU and acts as a DMA
(Direct Memory Access) controller, that transfers data between the hardware and a memory buffer.
A Xenomai real-time audio and sensor task can process data from this buffer and runs at a higher priority
than the Linux kernel.
So Pure Data's audio callback function "DSP_perform" runs as a Xenomai function at the highest priority.
https://bela.io/
Adafruit 9-DOF Absolute Orientation IMU head-tracker sensor: https://www.adafruit.com/product/2472
| QTY | PARTS |
|---|---|
| 1 | Bela Starter Kit |
| 1 | Adafruit 9-DOF Absolute Orientation IMU Fusion Breakout – BNO055 [ADA2472] |
| 1 | Breadboard |
| 2 | Push Buttons |
| 2 | Resistors 1.5k |
| 4 | Wire 1.5 m (SPI) |
| 4 | Wire 15 cm |
| 2 | Wire 5 cm |
| 1 | Headphones |
Connect Bela board via
- Ethernet to a network device that provides access to the internet.
- USB to a PC to power it on and to provide access to the onboard IDE through a webbrowser.
Determine the IP addresses assigned to the availiable network interfaces:
- Connect to the onboard Board IDE: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2/.
- In the command prompt at the bottom of the site type:
ip address
The output shows the IP adresses of all availiable network interfaces: see screenshot.
IPv4 addresses (inet) from the screenshot are:
- 3: usb0: 192.168.6.2 (for connection via USB)
- 2: eth0: 193.171.195.101 (for connection via Ethernet)
(IPv6 addresses will work as well.)
To access Bela's IDE Web-Interface open a webbrowser and go to http://bela.local/ or http://192.168.6.2/
Files can be copied to the Bela Board by simply drag-and-drop.
Access Bela via SFTP:
- IP address: 192.168.6.2
- Port: 22
- user: root
- no password
Possible SFTP applications are: Filezilla, putty, terminal,….
If access via SFTP is not working, check /etc/ssh/sshd_config. It must contain (uncommented):
PermitEmptyPasswords yes
UsePAM no
- https://github.com/BelaPlatform/Bela
- Click Clone or download and then Download ZIP
- Connect the Bela Board via Ethernet to the internet and via USB to the PC;
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2/.
- Click on the gearwheel icon on the top right bar to open the settings contextmenu.
- Scroll down the settings menu and click on the option Upload an update patch: Select the image file Bela-master.zip from the PC and click Upload
- Once the upload is complete, you will need to refresh the browser. Note: This may take several minutes!
An alternative installation method is to copy the image file Bela-master.zip from the PC to the directory ~/Bela/updates on the Bela board:
On a Linux PC open a terminal window and change to the directory, where the image file is stored:
cd <dir>
Copy the image file *Bela-master.zip from the PC to the dedicated directory on the Bela board with the secure copy command:
sudo scp ./Bela-master.zip [email protected]:~/Bela/updates
Alternatively use your favorite SFTP application with configuration as described above.
When the file upload is complete, connect from PC either to Bela's IDE via a webbrowser or use the ssh-client in the terminal window:
ssh [email protected]- confirm fingerprint with
yes
After gaining access to the Bela board change to the diretory, where the update script is located and run it:
cd ~/Bela/scriptssh update_board- confirm with
y
After updating the Bela Board, update its system files:
sudo apt-get updatesudo apt-get upgrade -y
Install gcovr:
sudo apt-get install -y gcovr
General remark: New projects are created in directory /root/Bela/projects on the Bela board. (File access via SFTP is possible)
- Connect the Bela Board via Ethernet to the internet and via USB to the PC.
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
- Click on the file icon on the top right bar to open the Project explorer contextmenu → click on New project → Enter project name: pd-externals (select any type).
- Delete all files from the directory ~/Bela/projects/pd-externals. (From IDE or via SFTP)
- Download the zip file from https://github.com/hoene/libmysofa to the PC.
- Click Clone or download and then Download ZIP.
- Extract the zip file to a convenient directory on the PC.
[remark miho: I used the version from P:\Projects\HRTF_measurement\SOFA\Software\SOFAlizer-for-pd\libmysofa-master 20200930]
- Connect the Bela Board via Ethernet to the internet and via USB to the PC.
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
- Click on the file icon on the top right bar to open the Project explorer contextmenu → click on New project → Enter project name: libmysofa (select any type).
- Delete all files from the directory /root/Bela/projects/libmysofa. (From IDE or via SFTP)
- Copy the extracted content from libmysofa.zip on the PC to the directory /root/Bela/projects/libmysofa on the Bela board via SFTP.
- Connect to the IDE and navigate in the command prompt to the directory ~/Bela/projects/libmysofa.
Then enter the following commands:
sudo apt install zlib1g-dev libcunit1-dev libcunit1-devcd buildcmake -DCMAKE_BUILD_TYPE=Release ..make all test
Note: This may take several minutes!
Don't worry, if some tests fail.
libmysofa libraries are created in /root/Bela/projects/libmysofa/build/src.
Static library libmysofa.a will be used for SOFAlizer later on!
- Download the zip file from https://github.com/sofacoustics/SOFAlizer-for-pd to the PC.
- Click Clone or download and then Download ZIP.
- Extract the zip file to a convenient directory called SOFAlizer-for-pd on the PC.
[remark miho: I used the version from P:\Projects\HRTF_measurement\SOFA\Software\SOFAlizer-for-pd\SOFAlizer-for-pd-master 20201012]
First Project:
- Connect the Bela Board via Ethernet to the internet and via USB to the PC.
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
- Click on the file icon on the top right bar to open the Project explorer contextmenu → click on New project → Enter project name: SOFAlizer_compile (select any type).
- Delete all files from the directory /root/Bela/projects/SOFAlizer_compile/. (From IDE or via SFTP)
- Copy/Upload the content from the subdirectory ./SOFAlizer~ of the directory ./SOFAlizer-for-pd on the PC to the directory /root/Bela/project/SOFAlizer_compile/ on the Bela board. (SFTP or IDE)
- Delete the library libmysofa.a from the directory /SOFAlizer_compile/
Copy the library libmysofa.a from the directory /root/Bela/projects/libmysofa/build/src (generated in chapter libmysofa) into the directory /root/Bela/projects/SOFAlizer_compile/ from command prompt: (overwrite if existing)
cp /root/Bela/projects/libmysofa/build/src/libmysofa.a /root/Bela/projects/SOFAlizer_compile/
Go to IDE and select in Project explorer (file icon on the top right bar) the Project SOFAlizer_compile.
Select Makefile and edit in IDE the addresses of the LDFLAGS and add at LIBS the library lgcov
Makefile (rows 34-35):
LDFLAGS += -L$(DIR)~/Bela/projects/SOFAlizer_compile -Wl,-R$(DIR)~/Bela/projects/SOFAlizer_compile '-Wl,-R$$ORIGIN'LIBS = libmysofa.a -lz -lgcov
Run Makefile from command prompt:
cd ~/Bela/projects/SOFAlizer_compilemake
The PD external SOFAlizer~.pd_linux is created. Copy SOFAlizer~.pd_linux to ~/Bela/projects/pd-externals:
cp ~/Bela/projects/SOFAlizer_compile/SOFAlizer~.pd_linux ~/Bela/projects/pd-externals
Second Project:
- Click on the file icon on the top right bar to open the Project explorer contextmenu → click on New project → Enter project name: SOFAlizer (select any type).
- Delete all files from the directory /root/Bela/projects/SOFAlizer/. (From IDE or via SFTP)
- Copy/Upload the content from the subdirectory ./bela_SOFAlizer of the directory ./SOFAlizer-for-pd on the PC to the directory /root/Bela/projects/SOFAlizer/ on the Bela board. (SFTP or IDE)
Note: An internal block size of 16 samples is not sufficient for this task.
The internal block size of Pure Date should be set to 64 samples, because this is the minimum block size Pure Data is processing data.
Even if the internal block size is set to 32 samples or to 16 samples, the actual delay of the filtering process remains the same regardless of the configured audio buffer size.
Setting a smaller internal block size than 64 samples in Bela means, that the libpd callback will effectively process data every four (with 16 samples per block) or two (with 32 samples per block) Bela audio callbacks.
Even though the minimum internal block size corresponds to the minimum audio buffer size, the lower latency achieved with an audio buffer size of 32 samples compared to a audio buffer size of 64 samples will make a difference of 1.437 ms.
- Connect the Bela Board via Ethernet to the internet and via USB to the PC.
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
Navigate to folder */root/Bela in command prompt:
cd ~/Belagit clone https://github.com/BelaPlatform/libpd.gitcd libpdgit submodule init(takes some time!)git submodule update --recursive(takes some time!)
Connect from PC via terminal to Bela board:
Open ~/Bela/libpd/pure-data/src/s_stuff.h
nano ~/Bela/libpd/pure-data/src/s_stuff.h
Scroll down to line #define DEFDACBLKSIZE and change block size from:
-
#define DEFDACBLKSIZE 16to -
#define DEFDACBLKSIZE 64 -
press
ctrl + xto exit -
type
yto confirm changes -
press
entertosave s_stuff.h -
type
exittoclose ssh connection -
Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
Navigate to folder ~/Bela/libpd from command prompt:
cd ~/Bela/libpd
Type:
-
make -f Makefile-Bela(Takes a couple of minutes! Last executed command from MAKEFILE takes longer.) -
make -f Makefile-Bela install(done afterif loop) -
In IDE click on the top right bar the gearwheel icon: Change Block size (audio frames) to the same value as PD's internal default block size: 64
- Mount tracker on top of the headphones
- Connect headphones to the audio output of the Bela board
- Connect an audio playback device to the audio input of the Bela board
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
- Click on the top right bar the file icon: Open existing project: SOFAlizer
- Select the file _main.pd
- Press
Run(It's the button with the refresh icon left at the bottom, just above the command prompt) - It takes ~3 minutes to read in two SOFA files until the audio processing is starting
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
- Click on the top right bar the gearwheel icon
- Scroll down to option Run project on boot
- Select: SOFAlizer
Type in command prompt:
shutdown --reboot now- When Bela Board reboots give it a few minutes to load the SOFA files
- When the project is running and the SOFA files have successfully been loaded, you should hear the audio signal, provided from your audio play-back device on your headphones
- Push the calibration button in the middle to reset the head-tracker position to 0°/0° twice or until the sound source is positioned directly infront of you.
- Start moving around your head and notice that the sound source stays in place and is perceived in the spatially correct position, according to your head position.
- To bypass spatialzation by SOFAlizer~ push the bypass button on top
- To switch between different HRTF sets defined in _main.pd push the switch button on the bottom
- Download the zip file from https://github.com/sofacoustics/SOFAlizer-for-pd to the PC.
- Click Clone or download and then Download ZIP.
- Extract the zip file to a convenient directory called SOFAlizer-for-pd on the PC.
- Connect the Bela Board via Ethernet to the internet and via USB to the PC.
- Connect to Bela's IDE Web-Interface: Open a webbrowser and go to http://bela.local/ or http://192.168.6.2.
- Click on the top right bar the file icon and open existing project: SOFAlizer_compile
- Drag and drop or press *Upload file" SOFAlizer~.c to the project window in browser and confirm to overwrite
- Upload any file that has changed in SOFAlizer-for-pd to its corresponding directory
If libmysofa.a has changed follow the steps in Install SOFA library libmysofa and copy libmysofa.a to /root/Bela/projects/SOFAlizer_compile/
cp /root/Bela/projects/libmysofa/build/src/libmysofa.a /root/Bela/projects/SOFAlizer_compile/- Edit addresses of LDFLAGS in Makefile if necessary
Run Makefile from command prompt:
cd ~/Bela/projects/SOFAlizer_compile/make
The PD external SOFAlizer~.pd_linux is created. Copy SOFAlizer~.pd_linux to ~/Bela/projects/pd-externals:
cp ~/Bela/projects/SOFAlizer_compile/SOFAlizer~.pd_linux ~/Bela/projects/pd-externals
Everything is set up now and you can run the _main.pd file from project SOFAlizer
- In IDE click on the top right bar the file icon and open existing project: SOFAlizer
- Select the _main.pd file
- Press button with refresh icon left on the bottom just above the command prompt to
runthe system - Enjoy!