Code

Use hdf5 references for arrays#118

Code

Merged

ka-sarthak merged 93 commits intowrite-nexus-sectionFAIRmat-NFDI/nomad-measurements:write-nexus-sectionfrom use-hdf5-references-for-arraysFAIRmat-NFDI/nomad-measurements:use-hdf5-references-for-arraysCopy head branch name to clipboardDec 19, 2024

+831-323

Commits

Commits on Dec 3, 2024

• updated plugin structure

  

  aalbino2

  authored and

  ka-sarthak

  committed
  cd342fd

• added pynxtools dependency

  

  aalbino2

  authored and

  ka-sarthak

  committed
  df6219b

• Apply suggestions from code review

  Show description for ce1e60b
  

  3 peoplecommitted
  ce1e60b

• Add sections for RSM and 1D which uses HDF5 references

  ka-sarthak

  committed
  de7b48e

• Abstract out data interaction using setter and getter; allows to use same methods for classes with hdf5 refs

  ka-sarthak

  committed
  47198b9

• Use arrays, not references, in the archive.results section

  ka-sarthak

  committed
  44ea11f

• Lock the state for using nexus file and corresponding references

  ka-sarthak

  committed
  6b66448

• Populate results without references

  ka-sarthak

  committed
  8d736b6

• Make a general reader for raw files

  ka-sarthak

  committed
  cd36d15

• Remove nexus flags

  ka-sarthak

  committed
  1d612f1

• Add quantity for auxialiary file

  ka-sarthak

  committed
  f796a79

• Fix rebase

  ka-sarthak

  committed
  6c59dad

Commits on Dec 4, 2024

• Make integration_time as hdf5reference

  ka-sarthak

  committed
  ad98a38

• Reset results (refactor)

  ka-sarthak

  committed
  1072dc0

• Add backward compatibility

  ka-sarthak

  committed
  65c8659

• Refactor reader

  ka-sarthak

  committed
  d5445ff

• add missing imports

  ka-sarthak

  committed
  6f16f44

• AttrDict class

  ka-sarthak

  committed
  d170d30

• Make concept map global

  ka-sarthak

  committed
  4f7bc83

• Add function to remove nexus annotations in concept map

  ka-sarthak

  committed
  a3295b7

• Move try block inside walk_through_object

  ka-sarthak

  committed
  89cdbc9

• Fix imports

  ka-sarthak

  committed
  54902f8

• Add methods for generating hdf5 file

  ka-sarthak

  committed
  393c572

• Rename auxiliary file

  ka-sarthak

  committed
  d2d1d1e

• Expect aux file to be .nxs in the beginning

  ka-sarthak

  committed
  30caf99

• Add attributes for hdf5: data_dict, dataset_paths

  ka-sarthak

  committed
  2f42f70

• Method for adding a quantity to hdf5_data_dict

  ka-sarthak

  committed
  2fef457

• Abstract out methods for creating files based on hdf5_data_dict

  ka-sarthak

  committed
  893d819

• Add dataset_paths for nexus

  ka-sarthak

  committed
  921a347

• Some reverting back

  ka-sarthak

  committed
  9f0cedf

• Minor fixes

  ka-sarthak

  committed
  f4fa2bd

Commits on Dec 5, 2024

• Refactor populate_hdf5_data_dict: store a reference to be made later

  ka-sarthak

  committed
  97a4083

• Handle shift from nxs to hdf5

  ka-sarthak

  committed
  191cd9d

• Set hdf5 references after aux file is created

  ka-sarthak

  committed
  8f4c4f4

• Cleaning

  ka-sarthak

  committed
  5f378d2

• Fixing

  ka-sarthak

  committed
  fdd2222

• Redefine result sections instead of extending

  ka-sarthak

  committed
  f49e168

Commits on Dec 6, 2024

• Remove plotly plots from ELN

  ka-sarthak

  committed
  f6f351c

• Read util for hdf5 ref

  ka-sarthak

  committed
  7cd535e

• Fixing

  ka-sarthak

  committed
  9d67aa2

• Move hdf5 handling into a util class

  ka-sarthak

  committed
  9ad1efe

• Refactor instance variables

  ka-sarthak

  committed
  8b8b9f4

• Reset data dicts and reference after each writing

  ka-sarthak

  committed
  382231b

• Fixing

  ka-sarthak

  committed
  f867a9a

• Overwrite dataset if it already exists

  ka-sarthak

  committed
  2b75a79

• Refactor add_dataset

  ka-sarthak

  committed
  97f4a3f

• Reorganize and doctrings

  ka-sarthak

  committed
  10bf72e

• Rename variable

  ka-sarthak

  committed
  b5b5666

• Add read_dataset method

  ka-sarthak

  committed
  88a9dde

• Cleaning

  ka-sarthak

  committed
  bbd0a86

• Adapting schema with hdf5 handler

  ka-sarthak

  committed
  94bdde7

• Cooments, minor refactoring

  ka-sarthak

  committed
  a859be2

• Fixing; add hdf5_handler as an attribute for archive

  ka-sarthak

  committed
  fea1724

• Reorganization

  ka-sarthak

  committed
  df6b571

• Fixing

  ka-sarthak

  committed
  ae96f5d

• Refactoring

  ka-sarthak

  committed
  e6cd6b8

• Cleaning

  ka-sarthak

  committed
  7b7db2b

Commits on Dec 7, 2024

• Try block for using hdf5 handler: dont fail early, as later normalization steps will have the handler!

  ka-sarthak

  committed
  11e4d38

• Extract units from dataset attrs when reading

  ka-sarthak

  committed
  de4f605

• Fixing

  ka-sarthak

  committed
  7f12438

• Linting

  ka-sarthak

  committed
  d7f69f7

Commits on Dec 9, 2024

• Make archive_path optional in add_dataset

  ka-sarthak

  committed
  be9dfa8

• Rename class

  ka-sarthak

  committed
  fb9f1c7

• attrs for add_dataset; use it for units

  ka-sarthak

  committed
  51915c3

• Add add_attribute method

  ka-sarthak

  committed
  090bc18

• Refactor add_attribute

  ka-sarthak

  committed
  6b2a95e

• Add plot attributes: 1D

  ka-sarthak

  committed
  3207e8e

Commits on Dec 10, 2024

• Refactor hdf5 states

  ka-sarthak

  committed
  dc3781c

• Add back plotly figures

  ka-sarthak

  committed
  24b4a9e

• rename auxiliary file name if changed by handler

  ka-sarthak

  committed
  17a0088

• Add referenced plots

  ka-sarthak

  committed
  c3302c4

• Allow hard link using internel reference

  ka-sarthak

  committed
  e993deb

• Add sections for plots

  ka-sarthak

  committed
  d753f4a

• Comment out validation

  ka-sarthak

  committed
  762fe86

Commits on Dec 11, 2024

• Add archive paths for the plot subsections

  ka-sarthak

  committed
  18ed92c

• Add back validation with flag

  ka-sarthak

  committed
  ff286c6

• Use nexus flag

  ka-sarthak

  committed
  374b6bb

Commits on Dec 13, 2024

• Add interpolated intensity data into h5 for qspace plots

  ka-sarthak

  committed
  9464f11

• Use prefix to reduce len of string

  ka-sarthak

  committed
  baac2da

• Store regularized linespace of q vectors; revise descriptions

  ka-sarthak

  committed
  d705957

• Remove plotly plots

  ka-sarthak

  committed
  47df0cc

• Bring plots to overview

  ka-sarthak

  committed
  fbc87d1

• Fix tests

  ka-sarthak

  committed
  c3f2ff3

• Linting; remove attr arg from add_dataset

  ka-sarthak

  committed
  3ba656f

Commits on Dec 16, 2024

• Review: move none check into method

  ka-sarthak

  committed
  4a0852c

Commits on Dec 17, 2024

• Review: use 'with' for opening h5 file

  ka-sarthak

  committed
  864d53b

• Review: make internal states as private vars

  ka-sarthak

  committed
  122b65f

• Add pydantic basemodel for dataset

  ka-sarthak

  committed
  67ee382

• Use data from variables if available for reading

  ka-sarthak

  committed
  645348a

Commits on Dec 18, 2024

• Review: remove lazy arg

  ka-sarthak

  committed
  64eae6c

• Move DatasetModel outside Handler class

  ka-sarthak

  committed
  e3164ff

Commits on Dec 19, 2024

• Remove None from get, as it is already a default

  ka-sarthak

  committed
  ac39a30

• Merge if conditions

  ka-sarthak

  committed
  3fa6263