Skip to content

Commit

Permalink
Save eiger metadata as a yaml string
Browse files Browse the repository at this point in the history
Because the numpy frame-cache format does not yet support nested
metadata structures, we cannot save nested metadata as a frame-cache
without causing errors.

We have some potential solutions to support nested metadata on the
horizon. Until that time, let's store the eiger stream metadata as
a yaml string, so that sparse arrays can actually be created from it.
The metadata is still present, it is just in the form of a yaml string.

Signed-off-by: Patrick Avery <[email protected]>
  • Loading branch information
psavery committed Aug 18, 2024
1 parent 47d5616 commit d9ba404
Showing 1 changed file with 17 additions and 1 deletion.
18 changes: 17 additions & 1 deletion hexrd/imageseries/load/eiger_stream_v1.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@
from dectris.compression import decompress
import h5py
import numpy as np
import yaml

Check warning on line 8 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L8

Added line #L8 was not covered by tests

from hexrd.utils.compatibility import h5py_read_string

Check warning on line 10 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L10

Added line #L10 was not covered by tests
from hexrd.utils.hdf5 import unwrap_h5_to_dict
from hexrd.utils.yaml import NumpyToNativeDumper

Check warning on line 12 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L12

Added line #L12 was not covered by tests

from . import ImageSeriesAdapter
from ..imageseriesiter import ImageSeriesIterator
Expand Down Expand Up @@ -96,8 +98,22 @@ def _load_metadata(self):

def _get_metadata(self):
d = {}
# First, unwrap the metadata from the HDF5 file into a dict
unwrap_h5_to_dict(self.__h5file['/metadata'], d)
return d

# Because frame cache imageseries do not yet support nested
# metadata structures, we should not use them. The frame cache
# imageseries should, at some point, start allowing for nested
# metadata structures. At that point, we can return to the
# previous way.
# For now instead, we will serialize this nested structure to
# yaml and keep it as a string. This means the metadata is still
# saved and accessible.
metadata = {

Check warning on line 112 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L112

Added line #L112 was not covered by tests
'eiger_metadata_as_yaml': yaml.dump(d, Dumper=NumpyToNativeDumper)
}

return metadata

Check warning on line 116 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L116

Added line #L116 was not covered by tests

@property
def metadata(self):
Expand Down

0 comments on commit d9ba404

Please sign in to comment.