Reading Kerchunk datasets

[tomwhite]

I created a Kerchunk dataset by following the tutorial here: https://fsspec.github.io/kerchunk/tutorial.html#combine-multiple-kerchunked-datasets-into-a-single-logical-aggregate-dataset

I was then able to read the dataset using Xarray and Cubed:

import fsspec
import ujson
import xarray as xr

from cubed import Spec

fs = fsspec.filesystem("")  # local file system to load json from

with fs.open("../kerchunk-example/air_pressure_at_mean_sea_level_combined.json", "rb") as f:
    d = ujson.load(f)

backend_args = {"consolidated": False, "storage_options": {"fo": d, "remote_protocol": "s3","remote_options": {"anon": True}}}

spec = Spec(work_dir="tmp", allowed_mem="1GB")

ds = xr.open_dataset(
    "reference://",
    engine="zarr",
    backend_kwargs=backend_args,
    chunked_array_type="cubed",
    from_array_kwargs={"spec": spec},
    chunks={},
)

print(ds)

Output:

<xarray.Dataset>
Dimensions:                         (time0: 1440, lat: 721, lon: 1440)
Coordinates:
  * lat                             (lat) float32 90.0 89.75 ... -89.75 -90.0
  * lon                             (lon) float32 0.0 0.25 0.5 ... 359.5 359.8
  * time0                           (time0) datetime64[ns] 2020-01-01 ... 202...
Data variables:
    air_pressure_at_mean_sea_level  (time0, lat, lon) float32 cubed.Array<chunksize=(24, 100, 100)>
Attributes:
    institution:  ECMWF
    source:       Reanalysis
    title:        ERA5 forecasts

It would be good to add this to the documentation, but in the meantime this issue will do.

[tomwhite]

Note that I needed cubed-dev/cubed#297 for the example to run.