Reading ADV Data with MHKiT
The following example demonstrates a simple workflow for analyzing Acoustic Doppler Velocimetry (ADV) data using MHKiT. MHKiT has brought the DOLfYN codebase in as an MHKiT module for working with ADV and Acoustic Doppler Current Profiler (ADCP) data. The DOLfYN MHKiT module is transferring functionality in two parts. In phase 1 the basic IO and core functions of DOLfYN are brought in. These functions include reading instrument data and rotating through coordinate systems. In phase 2 DOLfYN analysis functions are brought in. As phase 2 is still in progress we will import the analysis tools from the DOLfYN package. A typical ADV data workflow is broken down into
1. Review the raw data
- Check timestamps
- Look at velocity data quality, particularly for spiking
2. Check for spurious datapoints and remove. Replace bad datapoints using interpolation if desired
3. Rotate the data into principal flow coordinates (streamwise, cross-stream, vertical)
4. Average the data into bins, or ensembles, of a set time length (normally 5 to 10 min)
5. Calculate turbulence statistics (turbulence intensity, TKE, Reynolds stresses) of the measured flowfield
Read Raw Instrument Data
DOLfYN currently only carries support for the Nortek Vector ADV. The example loaded here is a short clip of data from a test deployment to show DOLfYN's capabilities.
Start by reading in the raw datafile downloaded from the instrument. The dolfyn_read function reads the raw file and dumps the information into an xarray Dataset, which contains three groups of variables:
1. Velocity, amplitude, and correlation of the Doppler velocimetry
2. Measurements of the instrument's bearing and environment
3. Orientation matrices DOLfYN uses for rotating through coordinate frames.
ds = dolfyn_read('data/dolfyn/vector_data01.VEC');
End of file at 3000000 bytes.
To see what's in the dataset, it is recommended that users open the ds variable in the workspace. Alternatively, removing the semicolon in the above line will print the contents of ds.
QC'ing Data
In future updates to MHKiT-Matlab's DOLfYN functionality users will be able to QC data and set certain helpful parameters such as deployment height.
Coordinate Rotations
The next step is to rotate the velocity data into true East, North, Up (ENU) coordinates.
ADVs use an internal compass or magnetometer to determine magnetic ENU directions. The set_declination function takes the user supplied magnetic declination (which can be looked up online for specific coordinates) and adjusts the orientation matrix saved within the dataset.
Instruments save vector data in the coordinate system specified in the deployment configuration file. To make the data useful, it must be rotated through coordinate systems ("beam"<->"inst"<->"earth"<->"principal"), done through the `rotate2` function. If the "earth" (ENU) coordinate system is specified, DOLfYN will automatically rotate the dataset through the necessary coordinate systems to get there.
% First set the magnetic declination
ds = set_declination(ds, 10.0);
% Rotate that data from the instrument to earth frame (ENU):
ds = rotate2(ds, 'earth');
Once in the true ENU frame of reference, we can calculate the principal flow direction for the velocity data and rotate it into the principal frame of reference (streamwise, cross-stream, vertical). Principal flow directions are aligned with and orthogonal to the flow streamlines at the measurement location.
First, the principal flow direction must be calculated through calc_principal_heading. As a standard for DOLfYN functions, those that begin with "calc_*" require the velocity data for input. This function is different from others in DOLfYN in that it requires that the user place the output in an attribute called "principal_heading", as shown below.
Again we use `rotate2` to change coordinate systems.
ds.attrs.principal_heading = calc_principal_heading(ds.vel.data, true);
ds = rotate2(ds, 'principal');
The next step in ADV analysis is to average the velocity data into time bins (ensembles) and calculate turbulence statistics. This and other features (such as the QC mentioned above will be in the next stage of DOLfYN development).
Saving and Loading DOLfYN Datasets
Datasets can be saved and loaded using the write_netcdf and read_netcdf functions, respectively.
% Uncomment these lines to save and load to your current working directory
% write_netcdf(ds, 'your_data.nc');
% ds_saved = read_netcdf('your_data.nc');
+.S8 { border-left: 1px solid rgb(217, 217, 217); border-right: 1px solid rgb(217, 217, 217); border-top: 0px none rgb(33, 33, 33); border-bottom: 1px solid rgb(217, 217, 217); border-radius: 0px 0px 4px 4px; padding: 0px 45px 4px 13px; line-height: 18.004px; min-height: 0px; white-space: nowrap; color: rgb(33, 33, 33); font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 14px; }
Reading ADV Data with MHKiT
The following example demonstrates a simple workflow for analyzing Acoustic Doppler Velocimetry (ADV) data using MHKiT. MHKiT has brought the DOLfYN codebase in as an MHKiT module for working with ADV and Acoustic Doppler Current Profiler (ADCP) data. The DOLfYN MHKiT module is transferring functionality in two parts. In phase 1 the basic IO and core functions of DOLfYN are brought in. These functions include reading instrument data and rotating through coordinate systems. In phase 2 DOLfYN analysis functions are brought in. As phase 2 is still in progress we will import the analysis tools from the DOLfYN package. A typical ADV data workflow is broken down into:
1. Review the raw data
- Check timestamps
- Look at velocity data quality, particularly for spiking
2. Check for spurious datapoints and remove. Replace bad datapoints using interpolation if desired
3. Rotate the data into principal flow coordinates (streamwise, cross-stream, vertical)
4. Average the data into bins, or ensembles, of a set time length (normally 5 to 10 min)
5. Calculate turbulence statistics (turbulence intensity, TKE, Reynolds stresses) of the measured flowfield
Read Raw Instrument Data
DOLfYN currently only carries support for the Nortek Vector ADV. The example loaded here is a short clip of data from a test deployment to show DOLfYN's capabilities.
Start by reading in the raw datafile downloaded from the instrument. The dolfyn_read function reads the raw file and dumps the information into an xarray Dataset, which contains three groups of variables:
1. Velocity, amplitude, and correlation of the Doppler velocimetry
2. Measurements of the instrument's bearing and environment
3. Orientation matrices DOLfYN uses for rotating through coordinate frames.
adv_file_path = fullfile(pwd, 'examples/data/dolfyn/vector_data01.VEC');
ds = dolfyn_read(adv_file_path);
End of file at 3000000 bytes.
To see what's in the dataset, it is recommended that users open the ds variable in the workspace. Alternatively, removing the semicolon in the above line will print the contents of ds.
QC'ing Data
In future updates to MHKiT-Matlab's DOLfYN functionality users will be able to QC data and set certain helpful parameters such as deployment height.
Coordinate Rotations
The next step is to rotate the velocity data into true East, North, Up (ENU) coordinates.
ADVs use an internal compass or magnetometer to determine magnetic ENU directions. The set_declination function takes the user supplied magnetic declination (which can be looked up online for specific coordinates) and adjusts the orientation matrix saved within the dataset.
Instruments save vector data in the coordinate system specified in the deployment configuration file. To make the data useful, it must be rotated through coordinate systems ("beam"<->"inst"<->"earth"<->"principal"), done through the `rotate2` function. If the "earth" (ENU) coordinate system is specified, DOLfYN will automatically rotate the dataset through the necessary coordinate systems to get there.
% First set the magnetic declination
ds = set_declination(ds, 10.0);
% Rotate that data from the instrument to earth frame (ENU):
ds = rotate2(ds, 'earth');
Once in the true ENU frame of reference, we can calculate the principal flow direction for the velocity data and rotate it into the principal frame of reference (streamwise, cross-stream, vertical). Principal flow directions are aligned with and orthogonal to the flow streamlines at the measurement location.
First, the principal flow direction must be calculated through calc_principal_heading. As a standard for DOLfYN functions, those that begin with "calc_*" require the velocity data for input. This function is different from others in DOLfYN in that it requires that the user place the output in an attribute called "principal_heading", as shown below.
Again we use `rotate2` to change coordinate systems.
ds.attrs.principal_heading = calc_principal_heading(ds.vel.data, true);
ds = rotate2(ds, 'principal');
The next step in ADV analysis is to average the velocity data into time bins (ensembles) and calculate turbulence statistics. This and other features (such as the QC mentioned above will be in the next stage of DOLfYN development).
Saving and Loading DOLfYN Datasets
Datasets can be saved and loaded using the write_netcdf and read_netcdf functions, respectively.
% Uncomment these lines to save and load to your current working directory
% write_netcdf(ds, 'your_data.nc');
% ds_saved = read_netcdf('your_data.nc');