Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sparsing from eiger stream files #709

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions hexrd/imageseries/load/eiger_stream_v1.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,11 @@
from dectris.compression import decompress
import h5py
import numpy as np
import yaml

Check warning on line 8 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L8

Added line #L8 was not covered by tests

from hexrd.utils.compatibility import h5py_read_string

Check warning on line 10 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L10

Added line #L10 was not covered by tests
from hexrd.utils.hdf5 import unwrap_h5_to_dict
from hexrd.utils.yaml import NumpyToNativeDumper

Check warning on line 12 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L12

Added line #L12 was not covered by tests

from . import ImageSeriesAdapter
from ..imageseriesiter import ImageSeriesIterator
Expand Down Expand Up @@ -95,8 +98,22 @@

def _get_metadata(self):
d = {}
# First, unwrap the metadata from the HDF5 file into a dict
unwrap_h5_to_dict(self.__h5file['/metadata'], d)
return d

# Because frame cache imageseries do not yet support nested
# metadata structures, we should not use them. The frame cache
# imageseries should, at some point, start allowing for nested
# metadata structures. At that point, we can return to the
# previous way.
# For now instead, we will serialize this nested structure to
# yaml and keep it as a string. This means the metadata is still
# saved and accessible.
metadata = {

Check warning on line 112 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L112

Added line #L112 was not covered by tests
'eiger_metadata_as_yaml': yaml.dump(d, Dumper=NumpyToNativeDumper)
}

return metadata

Check warning on line 116 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L116

Added line #L116 was not covered by tests

@property
def metadata(self):
Expand All @@ -116,7 +133,7 @@

@property
def dtype(self):
return self._first_data_entry['dtype'][()]
return h5py_read_string(self._first_data_entry['dtype'])

Check warning on line 136 in hexrd/imageseries/load/eiger_stream_v1.py

View check run for this annotation

Codecov / codecov/patch

hexrd/imageseries/load/eiger_stream_v1.py#L136

Added line #L136 was not covered by tests

@property
def shape(self):
Expand Down
8 changes: 6 additions & 2 deletions hexrd/utils/yaml.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,13 @@ class NumpyToNativeDumper(yaml.SafeDumper):
converted to a basic type.
"""
def represent_data(self, data):
if isinstance(data, np.ndarray):
# Empty shape arrays should be treated as numbers, not arrays.
is_empty_shape_array = (
isinstance(data, np.ndarray) and data.shape == ()
)
if isinstance(data, np.ndarray) and not is_empty_shape_array:
return self.represent_list(data.tolist())
elif isinstance(data, (np.generic, np.number)):
elif isinstance(data, (np.generic, np.number)) or is_empty_shape_array:
item = data.item()
if isinstance(item, (np.generic, np.number)):
# This means it was not converted successfully.
Expand Down
Loading