EchoDet
Welcome to EchoDet!
EchoDet is a piece of MATLAB software developed as a remora for Triton. It was created to detect high frequency, short duration pings produced by echosounders in acoustic datasets. The detector works by comparing a template echosounder timeseries (normalized waveform envelope) to that of raw data passed through the detector. Additional thresholds such as low and high frequency filter cutoffs, inter-pulse interval (i.e., the timing between successive echosounder pings) limits, and gap between detections can be used and modified to improve detector results. The remora includes a post-detection option of creating ID files for use in LabelVis.
Learn how to download or clone the Triton repository with the EchoDet Remora in the quick setup section and how to add the Remora to Triton.
This portion of EchoDet allows the user to specify load/save paths and thresholds for detection.
From the EchoDet drop down menu, choose ‘Run Echosounder Detector’.
This will open the main window for EchoDet, allowing the user to specify settings.
The load/save path section requires the following: Input Folder- File path to the input data (should be in format '.xwav')
Output Folder- Path to output directory where files will be saved
Echosounder Template- Path to template echosounder timeseries (should be a '.mat' file)
Transfer Function- Path to the transfer function used for your input files
Deployment Name- Prefix of the input data files; this is used to find input files and to name output files.
The echoDet remora comes with a built-in echosounder template, which can be found in the Remora files. To navigate to this template, use the following path:
(Triton save location) > Remoras > EchoDet > HI_echosounderTemplate.mat
This template is a timeseries extracted from an echosounder ping recorded by a High-Frequency Acoustic Recording Package (HARP) in the Hawaiian Islands. The template should be useable on data from other sources/regions as well. However, users may find that creating and using a template from an echosounder ping in their own data improves performance.
Thresholds for the echosounder detector can be modified to improve detector performance. Baseline settings for all thresholds were determined by the developer during detector testing based on maximizing detector performance on HARP data from the Hawaiian Islands.
Modifiable thresholds include the following:
Low Frequency Cutoff (Hz)- Low frequency cutoff in Hz for filtering the data (i.e., data below this frequency will not be considered by the detector).
High Frequency Cutoff (Hz)- Data above this frequency will not be considered by the detector.
Corr Thresh (perc) and Corr Thresh (Csq Value)- Thresholds for correlation between the template and incoming signals. The perc threshold specifies the percentile threshold input signals need to pass in order to be retained (i.e., for a given set of input signals with respective correlations to the template, only those above the median level of correlation within the set will be retained, for base settings). The Csq Value threshold sets the threshold for correlation similarity (squared) between the signal and template. The baseline for this is 50^2. These thresholds should not need to be modified.
ICI Range (sec)- Sets the allowable inter 'click' interval (i.e. timing between echosounder pulses, hereafter ICI) range for detections, in seconds. Detections with spacings outside of this range will be removed.
ICI Mode Pad (sec)- In addition to range limitations, detections of echosounders are pruned based on similarity to the modal ICI value in a given raw file. Mode pad sets the padding (in seconds) that you want to allow around the modal value for signals to be retained. With base settings, if the modal value was 2 seconds, allowable values would be from 1.9-2.1 seconds.
Detection gap (sec)- Allowable gap between detections. Currently set at 0.05 seconds, which helps mitigate echos of a given pulse being detected along with the original pulse.
RL Threshold- Received level threshold (dB peak to peak amplitude) between signal and preceding noise. Currently set to a very low value as to be effectively 'off'. However, may be helpful for some datasets. If desired, a value between 2-5 dBpp may be most effective.
When you have finished setting the appropriate paths and thresholds, press the 'run echosounder detector' button to run the detector. EchoDet will print a message in the MATLAB command window as it finishes each file.
When the detector is done running, it will print a message that says 'Done Running Echosounder Detector' to the command window. Output files will be stored in the specified output folder with the same naming conventions as input files and the suffix 'echo.mat'.
Output files contain the following:
dataSeg_final- Timeseries of all detections
noiseSeg_final- Timeseries of noise sample from directly before each detection
detDur_final- Duration of detected signal (MATLAB datenum format)
ED_stTimes_final- Start times for each detection (MATLAB datenum format)
f- Frequency vector corresponding to saved spectra (kHz)
keepCorr_final- Correlation squared values for each detection
keepWV_final- Waveform envelope of all detections
p- Structure with all settings used for the detector
spData_final- Spectra of all detections
spNoise_final- Spectra of noise sample from directly before each detection
ICI_final- ICI for all detections (seconds)
ICI_mode- Modal ICI for each raw file evaluated
ppSignal_final- Received level (dBpp) for all detections
The following are current options for post-processing available in EchoDet.
This functionality allows the user to make ID files that can be converted for use in LabelVis in order to evaluate detector performance. ID files refers to a file type ending in 'ID*.mat' which contain labels and times for detections in a dataset. In this case, an input folder of echo.mat files is converted into a single ID.mat file.
From the EchoDet drop down menu, choose ‘Create ID File Per Folder’.
The following window will appear.
Here, it is necessary to specify an input folder. One ID file will be made for this folder, combining all data files ending in 'echo.mat' found in that folder. The output ID file will be saved in the specified output folder location using the output file name, i.e. 'outputFileName_ID1.mat'.
If 'Run on subfolders' is checked, the following will appear.
In this case, the user will need to specify the subfolder prefix for all desired subfolders. Output files will be named after their corresponding subfolder, one file per folder.
In both cases, messages will be displayed in the MATLAB command window notifying the user upon completed processing of each file and folder. When all files have been completed, 'Done generating ID files' will appear.
Thank you for using EchoDet! Please address questions or comments to Morgan Ziegenhorn, [email protected]