cdf_to_xarray to_netcdf fails with ValueError

[berniegsfc]

The following code:

from cdflib.xarray import cdf_to_xarray,xarray_to_cdf
import xarray as xr

# https://cdaweb.gsfc.nasa.gov/pub/data/ace/mag/level_2_cdaweb/mfi_h0/1997/ac_h0_mfi_19970902_v04.cdf
filename = 'ac_h0_mfi_19970902_v04.cdf'

data = cdf_to_xarray(filename)

#xarray_to_cdf(data, 'data.cdf')

data.to_netcdf('data.nc')

fails with

ValueError: failed to prevent overwriting existing key units in attrs on variable 'Epoch'. This is probably an encoding field used by xarray to describe how a variable is serialized. To proceed, remove this key from the variable's attributes manually.

Is this a problem with the xarray.Dataset produced by cdf_to_xarray or a problem with xarray.Dataset.to_netcdf()?

[berniegsfc]

I've tried different versions of cdflib, xarray, and python and it fails the same in all. But for the record, the last test was with

• cdflib 1.3.1
• xarray 2024.7.0
• python 3.10.5

[bryan-harter]

Yeah the to_netcdf() command definitely doesn't like the "units" attribute when using np.datetime64 data. There is a couple ways I can see how to fix it:

1. You can ask cdflib not to convert the times to np.datetime64
   data = cdf_to_xarray(filename, to_datetime=False)

2. You can delete some of the items from the attributes before converting it:
   del data['Epoch'].attrs['units']
   or just don't include any attributes for the epoch variable:
   data['Epoch'].attrs = {}

Another issue it looks like is that the to_netcdf function doesn't like that FILLVAL/VALIDMIN/etc are all datetime64 objects. to_netcdf can convert datetime64 data over just fine, but I guess it cannot convert attribute values?

So the best course of action might be to not convert the times to datetime64. Or, if you do keep to_datetime=True, then you manually convert the attributes over to datetime64 objects if you want to keep them.

One thing I can do is add a flag to cdf_to_xarray that specifies if we should append "units", "standard_name", etc to the attributes. I have it do that automatically because it immediately allows nice plots of the data directly from xarray. But I'm open to other suggestions as well.