Esmval fixes

[ellesmith88]

There's a few fixes that I've added that I'm not 100% sure they're doing the same thing as the fixes in ESMValTool.

1. The mask data fix that I've written:

   daops/daops/data_utils/array_utils.py

   Lines 4 to 10 in 05f7fdb

   	def mask_data(ds, **operands):
   	value = operands.get("value")
   	var_id = xu.get_main_variable(ds)
   	ds = ds.where(ds[var_id] != value)
   	return ds

   
   equivalent ESMVal fix: https://github.com/ESMValGroup/ESMValCore/blob/ea6618c3fd014d32f29931853cc16c22453c4e09/esmvalcore/cmor/_fixes/cmip5/cesm1_bgc.py#L8-L26

Mine changes the values to NAN but I wondered whether it should return a masked array?

2. Reverse 2d vars fix and reverse coord fix:
   

   daops/daops/data_utils/var_utils.py

   Lines 4 to 15 in 29f139b

   	def reverse_2d_vars(ds, **operands):
   	var_ids = operands.get("var_ids")
   	for var_id in var_ids:
   	attrs = ds[var_id].attrs
   	dims = ds[var_id].dims
   	ds = ds.assign({f"{var_id}": (dims, ds[var_id].values[::-1, ::-1])})
   	ds[var_id].attrs = attrs
   	return ds

   
   

   daops/daops/data_utils/coord_utils.py

   Lines 35 to 43 in 29f139b

   	def reverse_coords(ds, **operands):
   	coords = operands.get("coords")
   	for coord in coords:
   	attrs = ds[coord].attrs
   	ds = ds.assign_coords({f"{coord}": ds[coord].values[::-1]})
   	ds[coord].attrs = attrs
   	return ds

   
   equivalent ESMVal fixes:
   https://github.com/ESMValGroup/ESMValCore/blob/ea6618c3fd014d32f29931853cc16c22453c4e09/esmvalcore/cmor/_fixes/cmip6/cesm2.py#L11-L76

I found their fix a bit confusing so it would be good to check I'm doing the right thing here

[agstephens]

@ellesmith88 this all looks good to me. (Sorry for taking so long).

The one thing that we might be able to generalise more is the reverse_2d_vars() function.

Could it be written as reverse_array_over_dimensions(var_ids=..., dims=...)

It could then build an arbitrary slice to reverse across whichever dimensions are specified.

E.g.:
Input data: rainfall (time[5], lat[10], lon[12])
Required fix: reverse time and lon
Input operands: {"var_ids": ["rainfall"], "dims": ["time", "lon"]}
And inside the code you do some kind of:

    slices = []
    dim_indices_to_reverse = .... # e.g. [0, 2]
    for i, dim in enumerate(ds[var_id].dims):
         if i not in dim_indices_to_reverse: 
             slices.append(slice(None, None))
         else:
             slices.append(slice(None, None, -1))

    ...then apply to the variable

NOTE: my understanding of Xarray dimension slicing might not be correct so please double-check.

[ellesmith88]

@agstephens At the moment I think this might take me too long to implement this, but I would like to come back to it. Should I include the fixes as they are in the new release of daops (and also merge the changes to dachar) or leave the PR open for now until I come back to it?

[agstephens]

@agstephens At the moment I think this might take me too long to implement this, but I would like to come back to it. Should I include the fixes as they are in the new release of daops (and also merge the changes to dachar) or leave the PR open for now until I come back to it?

@ellesmith88 I think we should progress this with the ESMValTool team rather than spending time refactoring at this stage. Are you happy to contact them to start the conversation?