Procedure to contribute model output to the portal

Add datasets to an existing THREDDS Data Server

For each simulation that appears on the portal, you need: (1) some NetCDF files in a directory, and (2) an NcML file that starts with “00_dir” (e.g. “00_dir_roms.ncml”) to aggregate, standardize and describe the dataset.

For ROMS datasets, you can use use the yaml2ncml python script to generate the NcML, using a YAML text file as input. Editing the YAML is simpler than editing the NcML directly because you don't have to edit attributes for multiple variables. You want the name of the resulting NcML file to start 00_dir and end with .ncml, like 00_dir_roms.ncml.

Here's an example YAML file and the ROMS NcML file that results.

Important: Use white space instead of tabs to indent the fields in your YAML file.

yaml2ncml is already installed on geoport and clay as part of the ioos python environment. To make this your default environment when you login to clay and/or geoport, add this line to your .bashrc:

export PATH=/home/usgs/miniconda/bin:$PATH && source activate IOOS

Then try logging out and back in, and you should see something like this:

discarding /home/usgs/miniconda/bin from PATH
prepending /home/usgs/miniconda/envs/IOOS/bin to PATH
(ioos)rsignell@clay:~$

which means that you are using the ioos environment.

Add the catalog that your NcML file is in to the list of thredds catalogs harvested by the catalog database. To do this, click the pen icon to edit, make your changes, and then click the button at the bottom of the page to submit a pull request.

When the pull request has been accepted, the python script on geoport then needs to be updated:

ssh geoport.whoi.edu
sudo su - usgs
cd /opt/pycsw/usgs-cmg-portal
git pull

The python script currently runs each hour at 20 past the hour. If everything is successful, there should be new iso metadata records that appear in /usgs/data0/iso/iso_records, and also in the pycsw database.