OpenCDMS Python library: pyopencdms

Overview

A Climate Data Management System (CDMS) is an integrated computer-based system that facilitates the effective archival, management, analysis, delivery and utilization of a wide range of integrated climate data (WMO 2014).

pyopencdms aims to build a common Python API supporting multiple Climate Data Management Systems (CDMS) that use different underlying database engines.

The image below shows the CliDE, Climsoft, MCH, MIDAS and other CDMSs being accessed through a single common API.

In addition we will add support for the WIGOS Meta Data Representation (WMDR) and collaborate with experts to create a new "CDMS Data Model Representation" that will support addional capabilities that are beyond the scope of WMDR.

The opencdms-test-data repository will be used as a source of test data for development to ensure interoperability between systems works as intended.

Dependencies

pyopencdms officially supports Python 3.7.1 and above, 3.8, and 3.9 (in line with the Pandas package)

pyopencdms uses SQLAlchemy (2.0-style) to connect to multiple database technologies including PostgreSQL, MySQL/MariaDB, Oracle and SQLite.

It is expected that SQLAlchemy objects, Panda's DataFrames and JSON will be key data types for exchanging data.

Example

• Create a virtual environment for OpenCDMS development
• Install dependencies used by pyopencdms
• Clone a copy of the opencdms-test-data repository

Example python commands

NOTE: The example below is old and will be updated by the end of 2021.

import os
from pathlib import Path

from opencdms import MidasOpen

# Instead of using a database connection string, the MIDAS Open
# provider requires the root directory for the MIDAS Open data.
connection = os.path.join(
    Path.home(), 'opencdms-dev', 'git', 'opencdms-test-data')

# All instances of CDMS Providers act as an active session
session = MidasOpen(connection)

filters = {
    'src_id': 838,
    'period': 'hourly',
    'year': 1991,
    'elements': ['wind_speed', 'wind_direction'],
}

# Get observations using filters
obs = session.obs(**filters)

# Save observations to CSV file
obs.to_csv('example_observations.csv')