Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Check: Dataset Attributes should be fixed-length strings #17

Open
nemo794 opened this issue Feb 28, 2025 · 1 comment
Open

Add Check: Dataset Attributes should be fixed-length strings #17

nemo794 opened this issue Feb 28, 2025 · 1 comment
Assignees

Comments

@nemo794
Copy link
Collaborator

nemo794 commented Feb 28, 2025

Currently, the QA XML checker reads in each Dataset's Attribute keys and values, only performing lightweight conversions if needed:

for key in hdf5_object.attrs.keys():
attribute = hdf5_object.attrs[key]
# Attributes are held in the
if type(attribute) in [bytearray, bytes, np.bytes_]:
attributes[key] = attribute.decode("utf-8")
else:
attributes[key] = attribute

This allows Attributes with variable-length string dtypes to pass through, and go un-logged. By NISAR conventions, all strings should be saved as as fixed-length bytes strings (or lists of fixed-length byte strings).

So, we need to add a check that string Attributes are either correctly fixed-length byte strings (or lists of fixed-length byte strings), or that an error is logged.

@nemo794 nemo794 self-assigned this Feb 28, 2025
@nemo794
Copy link
Collaborator Author

nemo794 commented Feb 28, 2025

fwiw, the R4.0.4 sample products (GSLC, GCOV, all InSAR) contain some Attributes which would fail this check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant