From 9440d9b2c046dc35c4b0c8709b8f6604ca96f6ad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20K=C3=BChbach?= Date: Thu, 30 Nov 2023 20:19:51 +0100 Subject: [PATCH] Base class templates (#51) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Refactored cg_primitive base classes for two key aspects: i) make all primitives inherit from new base class NXcg_primitive_set which makes a substantial number of repeated properties of specialized primitive base classes unnecessary, ii) introducing a rigorous mathematical set theoretical approach to describe the number type N, N0, Z, R, R+, R, iii) introduce short hand notation to reduce writing costs for shape/dimensions of arrays: now numpy syntax is used, for scalars (), (1, can be omitted, experience in FAIRmat has shown that in almost all cases the optional doc string to a dimension was almost never used and thus yaml based descriptions can be written more succinct, iv) proof-read docstrings and shortened them, introduced the suggestion to use info: as a keyword which should for now be just appended to the docstring as usual but in the future should be treated as a comment, i.e. only meant for human contextualization but ignored for semantic interpretation of concepts, a key difference that I hope this will bring is to reduce also the length of base classes and appdef as they show up in html pages, specifically for appdefs like NXem many of the conceptual ideas which these docstrings currently contain would be much better placed in a proper documentation instead of the actual class definition in the hope that the overall appearance of classes becomes easier and faster for people to power read; maybe this also works against some criticism that NeXus is considered complex, I have to say with the outsourcing of NXcg_primitive_set I feel it is damn simple now * Refactoring of NXcs base classes * First lot of refactoring EM base classes, the rest tomorrow, NXapm will not be refactored before the APT&M2023 conference in Sept2023, also because feedback from Leoben was positive enough that no immediate changes wrt to appdefs and base classes were required * Summarize the current state of the discussion about coordinate systems with the proposed NXcoordinate_system and NXcoordinate_system_set base classes and best practice recommendations how they should be used including feedback from discussions with Florian Dobner and Tamás Haraszti * Summary of the discussion how to handle conventions and method-specific conventions for EM and methods used in EM * Base classes for implemented examples for pole figures, orientation distribution function, event_data, stage_lab, and ebsd_crystal_structure_model * NXms_ipf base class and dos2unix to correct newlines * Introducing NXem_method base class as a template how to write method-specific deep base classes to describe the terms and taxononmy of method-specific branches to be hooked into appdefs like NXem * Base class inheritance proposal for a now even stronger modularized schema set for electron microscopy research, two tasks remain: i) refactor what is now a method-specific instance rather than an appdef (NXem_ebsd) (mainly to be able to fuse all examples of em-specific data converters including new ebsd examples into one em parser for pynxtools), ii) refactor NXem which is now clearly a specific appdef specializing the NXem_base deep base class, specialization work needs to define which fields and groups from all the possible ones now composable via base classes (inheritance) are required in an appdef NXem for NOMAD OASIS * NXem_ebsd refactored into a base class to use it as a method-specific group inside the NXem application definition, next step: i) refactor NXem_sim, ii) finalize refactoring of NXem appdef (for Nomad oasis) * finished draft of NXem_ebsd, NXem_correlation, and NXem appdef, cleaning the branch * Add proposal for storing mtx cfg, fixed nxtime datatypes * 2d microstructure projection * Inspection how proposed, info, N0, N, R, Z value type abbreviations, and dimensions could be added to nyamlforward * A likely too simplistic but at least working nyaml2nxdl forward mapping to explore further usage of refactored EM base classes. Info keyword has to be a child of doc or the respective text be removed from the standard and put into proposal-specific documentation, how to store what and where so that the schema docstring remain succinct and slim but all these conceptual ideas get not forgotten, typically the would be part of a tech report, i.e. in my opinion all what is under info: sections of a docstring should move to some documentation to tell the story to humans, next test these NXDLs with the NeXus documentation system * Minor fix to handle info keyword spotted while compiling the documentation * Fixes to compile with NeXus documentation test suite and sphinx * Deactivated the annoying clean yaml via make clean for dev purposes * Minor fix in em_base, this completes the appdef/base class work for now on the refactored EM, there are still some spurious info fields now, which should be removed when a decision has been made wrt to how to deal with info: keyword fields in general, next steps: i) make decision on info, ii) test refactored EM proposal with pynxtools em-parser v2, iii) implement backwards * Styling via black * Added yaml2nxdl-forward-converted NXDL files to all refactored base classes and the refactored NXem * Added NXroot header for the em appdef and its base template appdef * Continuing the refactoring of EM and APM plus related base classes for CG and MS based on suggestions from user meetings, discussions with Sandor, represents work with the MPES sprint #83 * Continuing on #83 * Continuing #83, NXcs_* * Continuing #83, ipf, pf, odf * Continuing on #83, support classes for EM * Continued on #83, coordinate system, further base classes supporting EM * Continuing #83, event_data_set and event_data description substantially condensed amongst other points * Added cross-references to base classes for rst, continuing #83 * Aligned old NXem_ebsd_conventions with NXcoordinate_system for #83 * Reviewed method-specific base classes, ebsd, eds, eels, #83 * #83, NXms_recon * #83, composed constraints on the NXem appdef * Consolidated with changes that happened in between on the fairmat branch based on 1016aa0, NXms_recon has still an issue and is therefore deactivated currently, method-specific landing pages need to be updated * Consolidated further with fairmat 15624c0be77be13a * Fixing some missing references * Fixed syntax error to compile NXms_recon, docs building also now, reviewing intro pages remains * Consistencies of dimensionality to use NX_POSINT and an enumeration * Recompiled NXDL files using new nyaml module 3d500ced7e4ca57683957c1d61a8d0cb62eccf53, removed, modified by taking the one from fairmat, and synced all files which were binarily different between this feature branch and the fairmat branch specifically commit a15798bab795d92b587527f2cff0819e26f550ee of the fairmat branch * Deactivated em-based tests which because of a refactoring of em are not expected to work anymore * Fix improper Latex notation in math environment for polyline, face_list, nanochem * Added recompiled NXidentifier, NXserialized NXDLs which triggered pipeline errors in CatChen gh action * Some round of proof-reading * Fixed test_nxdl_utils to reflect and use refactored locations of refactored NXem * Added feedback from @phyy-nx, @PeterC-DLS, and @prjemian from discussed here https://github.com/nexusformat/definitions PR #1271 * Black formatting * Reactivated data type check for e.g. NXem NX_NUMBER * Implementing NX_DATE_TIME suggestion of @sanbrock --------- Co-authored-by: markus.kuehbach # Conflicts: # applications/NXapm.nxdl.xml # applications/NXem.nxdl.xml # base_classes/NXaberration.nxdl.xml # base_classes/NXcg_alpha_complex.nxdl.xml # base_classes/NXcg_cylinder_set.nxdl.xml # base_classes/NXcg_face_list_data_structure.nxdl.xml # base_classes/NXcg_geodesic_mesh.nxdl.xml # base_classes/NXcg_grid.nxdl.xml # base_classes/NXcg_half_edge_data_structure.nxdl.xml # base_classes/NXcg_marching_cubes.nxdl.xml # base_classes/NXcg_polyline_set.nxdl.xml # base_classes/NXcg_triangulated_surface_mesh.nxdl.xml # base_classes/NXcg_unit_normal_set.nxdl.xml # base_classes/NXcoordinate_system_set.nxdl.xml # base_classes/NXcorrector_cs.nxdl.xml # base_classes/NXcs_computer.nxdl.xml # base_classes/NXcs_filter_boolean_mask.nxdl.xml # base_classes/NXcs_prng.nxdl.xml # base_classes/NXcs_profiling.nxdl.xml # base_classes/NXcs_profiling_event.nxdl.xml # base_classes/NXdeflector.nxdl.xml # base_classes/NXebeam_column.nxdl.xml # base_classes/NXem_ebsd.nxdl.xml # base_classes/NXevent_data_em.nxdl.xml # base_classes/NXevent_data_em_set.nxdl.xml # base_classes/NXibeam_column.nxdl.xml # base_classes/NXimage_set.nxdl.xml # base_classes/NXion.nxdl.xml # base_classes/NXlens_em.nxdl.xml # base_classes/NXoptical_system_em.nxdl.xml # base_classes/NXpump.nxdl.xml # base_classes/NXregistration.nxdl.xml # base_classes/NXscanbox_em.nxdl.xml # base_classes/NXspectrum_set.nxdl.xml # base_classes/NXstage_lab.nxdl.xml # contributed_definitions/NXaberration_model.nxdl.xml # contributed_definitions/NXaberration_model_ceos.nxdl.xml # contributed_definitions/NXaberration_model_nion.nxdl.xml # contributed_definitions/NXaperture_em.nxdl.xml # contributed_definitions/NXapm_paraprobe_results_nanochem.nxdl.xml # contributed_definitions/NXcg_ellipsoid_set.nxdl.xml # contributed_definitions/NXcg_hexahedron_set.nxdl.xml # contributed_definitions/NXcg_parallelogram_set.nxdl.xml # contributed_definitions/NXcg_point_set.nxdl.xml # contributed_definitions/NXcg_polygon_set.nxdl.xml # contributed_definitions/NXcg_polyhedron_set.nxdl.xml # contributed_definitions/NXcg_sphere_set.nxdl.xml # contributed_definitions/NXcg_tetrahedron_set.nxdl.xml # contributed_definitions/NXcg_triangle_set.nxdl.xml # contributed_definitions/NXchamber.nxdl.xml # contributed_definitions/NXcomponent_em.nxdl.xml # contributed_definitions/NXcs_gpu_sys.nxdl.xml # contributed_definitions/NXcs_io_obj.nxdl.xml # contributed_definitions/NXcs_io_sys.nxdl.xml # contributed_definitions/NXcs_mm_sys.nxdl.xml # contributed_definitions/NXem_adf.nxdl.xml # contributed_definitions/NXem_conventions.nxdl.xml # contributed_definitions/NXem_conventions_ebsd.nxdl.xml # contributed_definitions/NXem_ebsd_crystal_structure_model.nxdl.xml # contributed_definitions/NXgraph_edge_set.nxdl.xml # contributed_definitions/NXgraph_node_set.nxdl.xml # contributed_definitions/NXgraph_root.nxdl.xml # contributed_definitions/NXimage_r_set_diff.nxdl.xml # contributed_definitions/NXimage_set_em_adf.nxdl.xml # contributed_definitions/NXimage_set_em_kikuchi.nxdl.xml # contributed_definitions/NXinteraction_vol_em.nxdl.xml # contributed_definitions/NXisocontour.nxdl.xml # contributed_definitions/NXms.nxdl.xml # contributed_definitions/NXms_feature_set.nxdl.xml # contributed_definitions/NXms_ipf.nxdl.xml # contributed_definitions/NXms_mtex_config.nxdl.xml # contributed_definitions/NXms_odf.nxdl.xml # contributed_definitions/NXms_odf_set.nxdl.xml # contributed_definitions/NXms_pf.nxdl.xml # contributed_definitions/NXms_pf_set.nxdl.xml # contributed_definitions/NXms_recon.nxdl.xml # contributed_definitions/NXms_score_results.nxdl.xml # contributed_definitions/NXorientation_set.nxdl.xml # contributed_definitions/NXrotation_set.nxdl.xml # contributed_definitions/NXspectrum_set_em_eels.nxdl.xml # contributed_definitions/NXspectrum_set_em_xray.nxdl.xml # contributed_definitions/nyaml/NXaberration.yaml # contributed_definitions/nyaml/NXaberration_model.yaml # contributed_definitions/nyaml/NXaberration_model_ceos.yaml # contributed_definitions/nyaml/NXaberration_model_nion.yaml # contributed_definitions/nyaml/NXaperture_em.yaml # contributed_definitions/nyaml/NXapm.yaml # contributed_definitions/nyaml/NXapm_paraprobe_results_nanochem.yaml # contributed_definitions/nyaml/NXcg_alpha_complex.yaml # contributed_definitions/nyaml/NXcg_cylinder_set.yaml # contributed_definitions/nyaml/NXcg_ellipsoid_set.yaml # contributed_definitions/nyaml/NXcg_face_list_data_structure.yaml # contributed_definitions/nyaml/NXcg_geodesic_mesh.yaml # contributed_definitions/nyaml/NXcg_grid.yaml # contributed_definitions/nyaml/NXcg_half_edge_data_structure.yaml # contributed_definitions/nyaml/NXcg_hexahedron_set.yaml # contributed_definitions/nyaml/NXcg_marching_cubes.yaml # contributed_definitions/nyaml/NXcg_parallelogram_set.yaml # contributed_definitions/nyaml/NXcg_point_set.yaml # contributed_definitions/nyaml/NXcg_polygon_set.yaml # contributed_definitions/nyaml/NXcg_polyhedron_set.yaml # contributed_definitions/nyaml/NXcg_polyline_set.yaml # contributed_definitions/nyaml/NXcg_roi_set.yaml # contributed_definitions/nyaml/NXcg_sphere_set.yaml # contributed_definitions/nyaml/NXcg_tetrahedron_set.yaml # contributed_definitions/nyaml/NXcg_triangle_set.yaml # contributed_definitions/nyaml/NXcg_triangulated_surface_mesh.yaml # contributed_definitions/nyaml/NXcg_unit_normal_set.yaml # contributed_definitions/nyaml/NXchamber.yaml # contributed_definitions/nyaml/NXcoordinate_system_set.yaml # contributed_definitions/nyaml/NXcorrector_cs.yaml # contributed_definitions/nyaml/NXcs_computer.yaml # contributed_definitions/nyaml/NXcs_filter_boolean_mask.yaml # contributed_definitions/nyaml/NXcs_io_obj.yaml # contributed_definitions/nyaml/NXcs_io_sys.yaml # contributed_definitions/nyaml/NXcs_mm_sys.yaml # contributed_definitions/nyaml/NXcs_prng.yaml # contributed_definitions/nyaml/NXcs_profiling.yaml # contributed_definitions/nyaml/NXcs_profiling_event.yaml # contributed_definitions/nyaml/NXdeflector.yaml # contributed_definitions/nyaml/NXebeam_column.yaml # contributed_definitions/nyaml/NXem.yaml # contributed_definitions/nyaml/NXem_ebsd.yaml # contributed_definitions/nyaml/NXevent_data_em.yaml # contributed_definitions/nyaml/NXevent_data_em_set.yaml # contributed_definitions/nyaml/NXfabrication.yaml # contributed_definitions/nyaml/NXgraph_edge_set.yaml # contributed_definitions/nyaml/NXgraph_node_set.yaml # contributed_definitions/nyaml/NXgraph_root.yaml # contributed_definitions/nyaml/NXibeam_column.yaml # contributed_definitions/nyaml/NXimage_set.yaml # contributed_definitions/nyaml/NXinteraction_vol_em.yaml # contributed_definitions/nyaml/NXion.yaml # contributed_definitions/nyaml/NXisocontour.yaml # contributed_definitions/nyaml/NXlens_em.yaml # contributed_definitions/nyaml/NXms.yaml # contributed_definitions/nyaml/NXms_feature_set.yaml # contributed_definitions/nyaml/NXms_score_results.yaml # contributed_definitions/nyaml/NXoptical_system_em.yaml # contributed_definitions/nyaml/NXpump.yaml # contributed_definitions/nyaml/NXrotation_set.yaml # contributed_definitions/nyaml/NXscanbox_em.yaml # contributed_definitions/nyaml/NXspectrum_set.yaml # contributed_definitions/nyaml/NXstage_lab.yaml # dev_tools/tests/test_nxdl_utils.py # manual/source/classes/contributed_definitions/cgms-structure.rst # manual/source/classes/contributed_definitions/em-structure.rst # manual/source/classes/contributed_definitions/icme-structure.rst --- applications/NXapm.nxdl.xml | 1696 -------------- applications/NXem.nxdl.xml | 2036 ----------------- base_classes/NXaberration.nxdl.xml | 55 - base_classes/NXcg_alpha_complex.nxdl.xml | 6 +- base_classes/NXcg_cylinder_set.nxdl.xml | 6 +- .../NXcg_face_list_data_structure.nxdl.xml | 146 +- base_classes/NXcg_geodesic_mesh.nxdl.xml | 6 +- base_classes/NXcg_grid.nxdl.xml | 6 +- .../NXcg_half_edge_data_structure.nxdl.xml | 6 +- base_classes/NXcg_marching_cubes.nxdl.xml | 6 +- base_classes/NXcg_polyline_set.nxdl.xml | 6 +- .../NXcg_triangulated_surface_mesh.nxdl.xml | 6 +- base_classes/NXcg_unit_normal_set.nxdl.xml | 6 +- base_classes/NXcoordinate_system_set.nxdl.xml | 137 -- base_classes/NXcorrector_cs.nxdl.xml | 95 - base_classes/NXcs_computer.nxdl.xml | 80 - .../NXcs_filter_boolean_mask.nxdl.xml | 104 - base_classes/NXcs_prng.nxdl.xml | 85 - base_classes/NXcs_profiling.nxdl.xml | 149 -- base_classes/NXcs_profiling_event.nxdl.xml | 95 - base_classes/NXdeflector.nxdl.xml | 92 - base_classes/NXebeam_column.nxdl.xml | 109 - base_classes/NXem_ebsd.nxdl.xml | 1926 ---------------- base_classes/NXevent_data_em.nxdl.xml | 226 -- base_classes/NXibeam_column.nxdl.xml | 144 -- base_classes/NXion.nxdl.xml | 168 -- base_classes/NXlens_em.nxdl.xml | 109 - base_classes/NXoptical_system_em.nxdl.xml | 83 - base_classes/NXregistration.nxdl.xml | 55 - base_classes/NXscanbox_em.nxdl.xml | 47 - base_classes/NXspectrum_set.nxdl.xml | 162 -- base_classes/NXstage_lab.nxdl.xml | 173 -- .../NXaberration_model.nxdl.xml | 4 +- .../NXaberration_model_ceos.nxdl.xml | 4 +- .../NXaberration_model_nion.nxdl.xml | 4 +- .../NXapm_paraprobe_results_nanochem.nxdl.xml | 4 +- .../NXcg_ellipsoid_set.nxdl.xml | 135 ++ .../NXcg_hexahedron_set.nxdl.xml | 239 ++ .../NXcg_parallelogram_set.nxdl.xml | 183 ++ .../NXcg_point_set.nxdl.xml | 98 + .../NXcg_polygon_set.nxdl.xml | 225 ++ .../NXcg_polyhedron_set.nxdl.xml | 194 ++ .../NXcg_primitive_set.nxdl.xml | 212 ++ .../NXcg_sphere_set.nxdl.xml | 121 + .../NXcg_tetrahedron_set.nxdl.xml | 175 ++ .../NXcg_triangle_set.nxdl.xml | 132 ++ .../NXcomponent_em.nxdl.xml | 69 - .../NXcoordinate_system.nxdl.xml | 143 ++ .../NXcrystal_structure.nxdl.xml | 280 +++ contributed_definitions/NXcs_cpu_obj.nxdl.xml | 39 + ...gpu_sys.nxdl.xml => NXcs_cpu_sys.nxdl.xml} | 29 +- .../NXcs_gpu_obj.nxdl.xml | 27 +- contributed_definitions/NXcs_io_sys.nxdl.xml | 4 +- .../NXcs_mm_obj.nxdl.xml | 40 +- contributed_definitions/NXcs_mm_sys.nxdl.xml | 4 +- contributed_definitions/NXem_base.nxdl.xml | 389 ++++ .../NXem_conventions_ebsd.nxdl.xml | 230 -- .../NXem_correlation.nxdl.xml | 245 ++ contributed_definitions/NXem_eds.nxdl.xml | 144 ++ contributed_definitions/NXem_eels.nxdl.xml | 79 + .../{NXem_adf.nxdl.xml => NXem_img.nxdl.xml} | 32 +- contributed_definitions/NXem_method.nxdl.xml | 47 + contributed_definitions/NXem_msr.nxdl.xml | 96 + contributed_definitions/NXem_sim.nxdl.xml | 60 + .../NXgraph_node_set.nxdl.xml | 4 +- contributed_definitions/NXgraph_root.nxdl.xml | 4 +- .../NXimage_c_set.nxdl.xml | 247 ++ .../NXimage_r_set.nxdl.xml | 52 +- .../NXimage_r_set_diff.nxdl.xml | 179 -- .../NXinteraction_vol_em.nxdl.xml | 37 + contributed_definitions/NXisocontour.nxdl.xml | 4 +- contributed_definitions/NXms.nxdl.xml | 10 +- contributed_definitions/NXms_ipf.nxdl.xml | 383 ---- ...odf_set.nxdl.xml => NXms_ipf_set.nxdl.xml} | 14 +- .../NXms_mtex_config.nxdl.xml | 310 --- contributed_definitions/NXms_odf.nxdl.xml | 171 -- contributed_definitions/NXms_pf.nxdl.xml | 111 - contributed_definitions/NXms_recon.nxdl.xml | 454 ---- .../NXms_score_results.nxdl.xml | 22 +- .../{NXms_pf_set.nxdl.xml => NXroi.nxdl.xml} | 21 +- .../nyaml/NXcg_primitive_set.yaml | 136 ++ .../nyaml/NXcoordinate_system.yaml | 86 + .../nyaml/NXcrystal_structure.yaml | 178 ++ contributed_definitions/nyaml/NXem_base.yaml | 297 +++ .../nyaml/NXem_correlation.yaml | 191 ++ contributed_definitions/nyaml/NXem_eds.yaml | 80 + contributed_definitions/nyaml/NXem_eels.yaml | 39 + contributed_definitions/nyaml/NXem_img.yaml | 25 + .../nyaml/NXem_method.yaml | 21 + contributed_definitions/nyaml/NXem_msr.yaml | 63 + contributed_definitions/nyaml/NXem_sim.yaml | 34 + contributed_definitions/nyaml/NXroi.yaml | 9 + 92 files changed, 4908 insertions(+), 9987 deletions(-) delete mode 100644 applications/NXapm.nxdl.xml delete mode 100644 applications/NXem.nxdl.xml delete mode 100644 base_classes/NXaberration.nxdl.xml delete mode 100644 base_classes/NXcoordinate_system_set.nxdl.xml delete mode 100644 base_classes/NXcorrector_cs.nxdl.xml delete mode 100644 base_classes/NXcs_computer.nxdl.xml delete mode 100644 base_classes/NXcs_filter_boolean_mask.nxdl.xml delete mode 100644 base_classes/NXcs_prng.nxdl.xml delete mode 100644 base_classes/NXcs_profiling.nxdl.xml delete mode 100644 base_classes/NXcs_profiling_event.nxdl.xml delete mode 100644 base_classes/NXdeflector.nxdl.xml delete mode 100644 base_classes/NXebeam_column.nxdl.xml delete mode 100644 base_classes/NXem_ebsd.nxdl.xml delete mode 100644 base_classes/NXevent_data_em.nxdl.xml delete mode 100644 base_classes/NXibeam_column.nxdl.xml delete mode 100644 base_classes/NXion.nxdl.xml delete mode 100644 base_classes/NXlens_em.nxdl.xml delete mode 100644 base_classes/NXoptical_system_em.nxdl.xml delete mode 100644 base_classes/NXregistration.nxdl.xml delete mode 100644 base_classes/NXscanbox_em.nxdl.xml delete mode 100644 base_classes/NXspectrum_set.nxdl.xml delete mode 100644 base_classes/NXstage_lab.nxdl.xml create mode 100644 contributed_definitions/NXcg_ellipsoid_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_hexahedron_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_parallelogram_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_point_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_polygon_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_polyhedron_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_primitive_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_sphere_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_tetrahedron_set.nxdl.xml create mode 100644 contributed_definitions/NXcg_triangle_set.nxdl.xml delete mode 100644 contributed_definitions/NXcomponent_em.nxdl.xml create mode 100644 contributed_definitions/NXcoordinate_system.nxdl.xml create mode 100644 contributed_definitions/NXcrystal_structure.nxdl.xml create mode 100644 contributed_definitions/NXcs_cpu_obj.nxdl.xml rename contributed_definitions/{NXcs_gpu_sys.nxdl.xml => NXcs_cpu_sys.nxdl.xml} (65%) rename base_classes/NXevent_data_em_set.nxdl.xml => contributed_definitions/NXcs_gpu_obj.nxdl.xml (61%) rename base_classes/NXpump.nxdl.xml => contributed_definitions/NXcs_mm_obj.nxdl.xml (56%) create mode 100644 contributed_definitions/NXem_base.nxdl.xml delete mode 100644 contributed_definitions/NXem_conventions_ebsd.nxdl.xml create mode 100644 contributed_definitions/NXem_correlation.nxdl.xml create mode 100644 contributed_definitions/NXem_eds.nxdl.xml create mode 100644 contributed_definitions/NXem_eels.nxdl.xml rename contributed_definitions/{NXem_adf.nxdl.xml => NXem_img.nxdl.xml} (75%) create mode 100644 contributed_definitions/NXem_method.nxdl.xml create mode 100644 contributed_definitions/NXem_msr.nxdl.xml create mode 100644 contributed_definitions/NXem_sim.nxdl.xml create mode 100644 contributed_definitions/NXimage_c_set.nxdl.xml rename base_classes/NXimage_set.nxdl.xml => contributed_definitions/NXimage_r_set.nxdl.xml (61%) delete mode 100644 contributed_definitions/NXimage_r_set_diff.nxdl.xml create mode 100644 contributed_definitions/NXinteraction_vol_em.nxdl.xml delete mode 100644 contributed_definitions/NXms_ipf.nxdl.xml rename contributed_definitions/{NXms_odf_set.nxdl.xml => NXms_ipf_set.nxdl.xml} (80%) delete mode 100644 contributed_definitions/NXms_mtex_config.nxdl.xml delete mode 100644 contributed_definitions/NXms_odf.nxdl.xml delete mode 100644 contributed_definitions/NXms_pf.nxdl.xml delete mode 100644 contributed_definitions/NXms_recon.nxdl.xml rename contributed_definitions/{NXms_pf_set.nxdl.xml => NXroi.nxdl.xml} (72%) create mode 100644 contributed_definitions/nyaml/NXcg_primitive_set.yaml create mode 100644 contributed_definitions/nyaml/NXcoordinate_system.yaml create mode 100644 contributed_definitions/nyaml/NXcrystal_structure.yaml create mode 100644 contributed_definitions/nyaml/NXem_base.yaml create mode 100644 contributed_definitions/nyaml/NXem_correlation.yaml create mode 100644 contributed_definitions/nyaml/NXem_eds.yaml create mode 100644 contributed_definitions/nyaml/NXem_eels.yaml create mode 100644 contributed_definitions/nyaml/NXem_img.yaml create mode 100644 contributed_definitions/nyaml/NXem_method.yaml create mode 100644 contributed_definitions/nyaml/NXem_msr.yaml create mode 100644 contributed_definitions/nyaml/NXem_sim.yaml create mode 100644 contributed_definitions/nyaml/NXroi.yaml diff --git a/applications/NXapm.nxdl.xml b/applications/NXapm.nxdl.xml deleted file mode 100644 index 6f02ae1809..0000000000 --- a/applications/NXapm.nxdl.xml +++ /dev/null @@ -1,1696 +0,0 @@ - - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Total number of ions collected. - - - - - Total number of independent wires in the delay-line detector. - - - - - Number of support points for e.g. modeling peaks. - - - - - Maximum number of allowed atoms per (molecular) ion (fragment). - Needs to match maximum_number_of_atoms_per_molecular_ion. - - - - - Number of mass-to-charge-state-ratio intervals of this ion type. - - - - - Number of bins in the x direction. - - - - - Number of bins in the y direction. - - - - - Number of bins in the z direction. - - - - - Number of bins. - - - - - Total number of integers in the supplementary XDMF topology array. - - - - - Application definition for atom probe and field ion microscopy experiments. - - This application definition provides a place to document data and metadata to - an atom probe experiment. Primarily the measurement itself is documented. - However, as most atom probe experiments are controlled with commercial software - which does not allow to access the raw detector hits, this application definition - also includes two key groups of processing steps (reconstruction and ranging). - - During tomographic reconstruction measured data are processed into a point cloud - of reconstructed positions of certain ions. During ranging time-of-flight data - are identified as representing specific ions to annotate each ion with a label. - - Commercial software used in atom probe research is designed as an integrated - acquisition and instrument control software. For AMETEK/Cameca local electrode - atom probe (LEAP) instruments the least processed (rawest) numerical results - and metadata are stored in so-called STR, RRAW, RHIT, and HITS files, which - are proprietary and their file format specifications not publicly documented. - - Supplementary metadata are kept in a database (formerly known as the ISDb) - which is connected to the instrument control software and synced with the - experiment while ions are detected. In effect, RHIT and HITS files - store the (rawest) experiment data in a closed manner that is - practically useless for users unless they have access to the - commercial software. - - To arrive at a state that atom probe microscopy (APM) with LEAP instruments - delivers a dataset with which users can study reconstructed atomic - position and do e.g. composition analyses or other post-processing - analysis tasks, these raw data have to be processed. Therefore, it is - necessary that for an application definition to be useful, details about - the physical acquisition of the raw data and all its - processing steps have to be stored. - - With this a user can create derived quantities like ion hit positions - (on the detector) and calibrated time-of-flight data. These derived - quantities are also needed to obtain calibrated mass-to-charge-state - ratios, and finally the tomographic reconstruction of the ion positions. - - In most cases, an APM dataset is useful only if it gets post-processed - via so-called ranging. Ranging defines rules for mapping time-of-flight - and mass-to-charge-state ratio values on ion species. This is post-processing - even though in practice it is performed sometimes already (as preview) - already while data are still being collected. - - The ion types decode molecular identities which can very often be - mapped to elemental identities, and also be used to distinguish isotopes. - All these steps are in most cases performed using commercial software. - - Frequently, though, ranging and post-processing is also performed with - (open-source) research software. Therefore, there is strictly speaking - not a single program used throughout an atom probe analysis not even - for the early stage of data acquisition and processing stages to obtain - a useful reconstructed and ranged dataset. - - This application definition documents not only the measurement but also the - key post-processing steps which transform the proprietary data into a - tomographic reconstruction with ranging definitions. - - Future guidance by the technology partners like AMETEK/Cameca could improve - this description to cover a substantial larger number of eventually metadata - that so far are neither publicly documented nor accessible. - - - - - An at least as strong as SHA256 hashvalue of the file - that specifies the application definition. - - - - - - NeXus NXDL schema to which this file conforms. - - - - - - - - Ideally, a (globally) unique persistent identifier - for referring to this experiment. - - The identifier is usually defined/issued by the facility, laboratory, - or the principle investigator. The identifier enables to link - experiments to e.g. proposals. - - - - - Free-text description about the experiment. - - Users are strongly advised to detail the sample history in the - respective field and fill rather as completely as possible the fields - of the application definition behind instead of filling in these - details into the experiment_description free-text description field. - - Users are encouraged to add in this field eventual DOIs to papers - which yield further details to the experiment. - - - - - ISO 8601 time code with local time zone offset to UTC information - included when the microscope session started. - If the application demands that time codes in this section of the - application definition should only be used for specifying when the - experiment was performed - and the exact duration is not relevant - - this start_time field should be used. - - Often though it is useful to specify a time interval with specifying - both start_time and end_time to allow for more detailed bookkeeping - and interpretation of the experiment. The user should be aware that - even with having both dates specified, it may not be possible - to infer how long the experiment took or for how long data - were collected. - - More detailed timing data over the course of the experiment have to be - collected to compute this event chain during the experiment. - - - - - - ISO 8601 time code with local time zone offset to UTC included - when the microscope session ended. - - - - - - - - - - - Neither the specimen_name nor the experiment_identifier but the identifier - through which the experiment is referred to in the control software. - For LEAP instruments it is recommended to use the IVAS/APSuite - run_number. For other instruments, such as the one from Stuttgart or - Oxcart from Erlangen, or the instruments at GPM in Rouen, use the - identifier which is closest in meaning to the LEAP run number. - The field does not have to be required if the information is recoverable - in the dataset which for LEAP instruments is the case when RHIT or HITS - files are also stored alongside a data artifact instance which is - generated according to this NXapm application definition. - - As a destructive microscopy technique, a run can be performed only once. - It is possible, however, to interrupt a run and restart data acquisition - while still using the same specimen. In this case, each evaporation run - needs to be distinguished with different run numbers. - We follow this habit of most atom probe groups. - - This application definition does currently not allow storing the - entire set of such interrupted runs. Not because of a technical limitation - within NeXus but because we have not seen a covering use case based - on which we could have designed and implemented this case. - Atom probers are invited to contact the respective people in the - FAIRmat team to fix this. - - - - - Binary container for a file or a compressed collection of files which - can be used to add further descriptions and details to the experiment. - The container can hold a compressed archive. - - Required for operation_mode apt_fim or other to give further details. - Users should not abuse this field to provide free-text information. - Instead, these pieces of information should be mapped to - respective groups and sections. - - - - - A small image that is representative of the entry; this can be an - image taken from the dataset like a thumbnail of a spectrum. - A 640 x 480 pixel jpeg image is recommended. - Adding a scale bar to that image is recommended but not required - as the main purpose of the thumbnail is to provide e.g. thumbnail - images for displaying them in data repositories. - - - - - - What type of atom probe microscopy experiment is performed. - This field is primarily meant to inform materials database systems - to qualitatively filter experiments: - - * apt are experiments where the analysis_chamber has no imaging gas. - experiment with LEAP instruments are typically performed as apt. - * fim are experiments where the analysis_chamber has an imaging gas, - which should be specified with the atmosphere in the analysis_chamber group. - * apt_fim should be used for combinations of the two imaging modes. - * other should be used in combination with the user specifying details in the - experiment_documentation field. - - - - - - - - - - - - Contact information and eventually details person(s) involved in the - microscope session. This can be the principle investigator who performed - this experiment. Adding multiple users if relevant is recommended. - - - - Given (first) name and surname of the user. - - - - - Name of the affiliation of the user at the point in time - when the experiment was performed. - - - - - Postal address of the affiliation. - - - - - Email address of the user at the point in time when the experiment - was performed. Writing the most permanently used email is recommended. - - - - - Globally unique identifier of the user as offered by services - like ORCID or ResearcherID. If this field is field the specific - service should also be written in orcid_platform - - - - - Name of the OrcID or ResearcherID where the account - under orcid is registered. - - - - - (Business) (tele)phone number of the user at the point - in time when the experiment was performed. - - - - - Which role does the user have in the place and at the point - in time when the experiment was performed? Technician operating - the microscope. Student, postdoc, principle investigator, guest - are common examples. - - - - - Account name that is associated with the user - in social media platforms. - - - - - Name of the social media platform where the account - under social_media_name is registered. - - - - - - Description of the sample from which the specimen was prepared or - site-specifically cut out using e.g. a focused-ion beam instrument. - - The sample group is currently a place for storing suggestions from - atom probers about other knowledge they have gained about the sample - from which they cut the specimen which is field-evaporated during the - experiment. Typically this is possible because the atom probe specimen - is usually not heat treated as is but one assumes that one has the sample - prepared as needed (i.e. with a specific grain diameter) and can thus - just cut out the specimen from that material. - - There are cases though where the specimen is processed further, i.e. the - specimen is machined further or exposed to external stimuli during the - experiment. In this case, these details should not be stored in the - sample group but atom probers should make suggestions how this application - definition can be improved to find a better place and compromise - how to improve this application definition. - - In the future also details like how the grain_diameter was characterized, - how the sample was prepared, how the material was heat-treated etc., - should be stored as using specific application definitions/schemas - which are then arranged and documented with a description of the workflow - so that actionable graphs become instantiatable. - - - - Qualitative information about the grain size, here specifically - described as the equivalent spherical diameter of an assumed - average grain size for the crystal ensemble. - Users of this information should be aware that although the grain - diameter or radius is often referred to as grain size and used in - e.g. Hall-Petch-type materials models this is if at all an ensemble - average whose reporting may be very informative or not if the specimen - contains a few grains only. In atom probe the latter is often the case - because grains are measured partially as their diameter can be in the - order of magnitude of the physical diameter of the specimen. - - Reporting a grain size is useful though as it allows judging if - specific features are expected to be found in the detector hit map. - - - - - Magnitude of the standard deviation of the grain_diameter. - - - - - The temperature of the last heat treatment step before quenching. - Knowledge about this value can give an idea how the sample - was heat treated, however if available a documentation of the - annealing treatment should be delivered by adding additional files - which are uploaded alongside an NXapm instance. - In the future there should better be an own schema used for the - heat treatment. - - - - - Magnitude of the standard deviation of the heat_treatment_temperature. - - - - - Rate of the last quenching step. - Knowledge about this value can give an idea how the specimen - was heat treated, however there are many situations where one - can imagine that the scalar value for just the quenching rate, - i.e. the first derivative of the measured time-temperature profile - is itself time-dependant. An example is when the specimen was - left in the furnace after the furnace was switched off. In this case - the specimen cools down with a specific rate of how this furnace - cools down in the lab. Processes which in practice are often not - documented with measuring the time-temperature profile. - - This can be problematic because when the furnace door was left open - or the ambient temperature in the lab changes, i.e. for a series of - experiments where one is conducted on a hot summer - day and the next during winter as might have an effect on the - evolution of the microstructure. There are many cases where this - has been reported to be an issue in industry, e.g. think about aging - aluminium samples left on the factory parking lot on a hot summer - day. - - - - - Magnitude of the standard deviation of the heat_treatment_quenching_rate. - - - - - - The chemical composition of the sample. Typically it is assumed that - this more macroscopic composition is representative for the material - so that the composition of the typically substantially less voluminous - specimen probes from the more voluminous sample. - - - - Reporting compositions as atom and weight percent yields both - dimensionless quantities but their conceptual interpretation - differs. A normalization based on atom_percent counts relative to the - total number of atoms are of a particular type. By contrast, weight_percent - normalization factorizes in the respective mass of the elements. - Python libraries like pint are challenged by these differences as - at.-% and wt.-% both yield fractional quantities. - - - - - - - - - - Human-readable name of the element/ion (e.g. Fe). - Name has to be a symbol of an element from the periodic table. - All symbols in the set of NXion instances inside the group - chemical_composition need to be disjoint. - - - - - Composition value for the element/ion referred to under name. - The value is normalized based on normalization, i.e. composition - is either an atom or weight percent quantity. - - - - - Magnitude of the standard deviation of the composition (value). - - - - - - - - - - Descriptive name or ideally (globally) unique persistent identifier. - The name distinguishes the specimen from all others and especially the - predecessor/origin (see the sample group) from where this specimen was cut. - In cases where the specimen was e.g. site-specifically cut from the - sample referred to in the sample group or in cases of an instrument session - during which multiple specimens are loaded, the name has to be descriptive - enough to resolve which specimen on e.g. the microtip array was taken. - - The user is advised to store the details how a specimen was cut/prepared - from a specific sample in the sample_history. The sample_history field - must not be used for writing an alias of the specimen. Instead, - use the field alias for this. As an example there may be a specimen/sample - monitoring system in a lab with bar codes. The bar code is a good - specimen/sample name. A shorter and more human readable alias like - A0 can be an example for something to write in the alias field. - - In cases where multiple specimens have been loaded into the microscope - the name has to be the specific one, whose results are stored - by this NXentry, because a single NXentry is to be used for the - characterization of a single specimen in a single continuous run. - - Details about the specimen preparation should be stored in the - sample_history or if this is not possible in the sample group. - - - - - Ideally, a reference to the location of or a (globally) unique - persistent identifier of e.g. another file which should document - ideally as many details as possible of the material, its - microstructure, and its thermo-chemo-mechanical processing/ - preparation history. - - In the case that such a detailed history of the sample/specimen is not - available, use this field as a free-text description to specify a - sub-set of the entire sample history, i.e. what you would consider - as being the key steps and relevant information about the specimen, - its material, microstructure, thermo-chemo-mechanical processing - state and details of the preparation. - - - - - ISO 8601 time code with local time zone offset to UTC information - when the specimen was prepared. - - Ideally, report the end of the preparation, i.e. the last known time - the measured specimen surface was actively prepared. Usually this - should be a part of the sample history, i.e. the sample is imagined - handed over for the analysis. At the point it enters the microscope - the session starts. - - Knowing when the specimen was exposed to e.g. specific atmosphere is - especially required for environmentally sensitive material such as - hydrogen charged specimens or experiments including tracers with a - short half time. Further time stamps prior to preparation_date should - better be placed in resources which describe the sample_history. - - - - - Short_name or abbreviation of the specimen name field. - - - - - List of comma-separated elements from the periodic table - that are contained in the sample. - If the sample substance has multiple components, all - elements from each component must be included in `atom_types`. - - The purpose of the field is to offer materials database systems an - opportunity to parse the relevant elements without having to interpret - these from the sample history or from other data sources. - - - - - Discouraged free-text field in case properly designed records - for the sample_history or sample section are not available. - - - - - Report if the specimen is polycrystalline, in which case it - contains a grain or phase boundary, or if the specimen is a - single crystal. - - - - - - - Hard link to a location in the hierarchy of the NeXus file - where the data for default plotting are stored. - - - - - Container to hold different coordinate systems conventions. - - For the specific idea and conventions to use with the - NXcoordinate_system_set inspect the description of the - NXcoordinate_system_set base class. Specific details for application - in atom probe microscopy follow. - - In this research field scientists distinguish usually several - Euclidean coordinate systems (CS): - - * World space; - a CS specifying a local coordinate system of the planet earth which - identifies into which direction gravity is pointing such that - the laboratory space CS can be rotated into this world CS. - * The laboratory space; - a CS specifying the room where the instrument is located in or - a physical landmark on the instrument, e.g. the direction of the - transfer rod where positive is the direction how the rod - has to be pushed during loading a specimen into the instrument. - In summary, this CS is defined by the chassis of the instrument. - * The specimen space; - a CS affixed to either the base or the initial apex of the specimen, - whose z axis points towards the detector. - * The detector space; - a CS affixed to the detector plane whose xy plane is usually in the - detector and whose z axis points towards the specimen. - This is a distorted space with respect to the reconstructed ion - positions. - * The reconstruction space; - a CS in which the reconstructed ion positions are defined. - The orientation depends on the analysis software used. - * Eventually further coordinate systems attached to the - flight path of individual ions might be defined. - - Coordinate systems should be right-handed ones. - Clockwise rotations should be considered positive rotations. - - In atom probe microscopy a frequently used choice for the detector - space (CS) is discussed with the so-called detector space image - (stack). This is a stack of two-dimensional histograms of detected ions - within a predefined evaporation ID interval. Typically, the set of - ion evaporation sequence IDs is grouped into chunks. - - For each chunk a histogram of the ion hit positions on the detector - is computed. This leaves the possibility for inconsistency between - the so-called detector space and the e.g. specimen space. - - The transformation here resolves this ambiguity by specifying how - the positive z-axes of either coordinate systems is oriented. - Consult the work of A. J. Breen and B. Gault and team - for further details. - - - - - - - - - - Metadata and numerical data of the atom probe and the lab in which it - stands. - - An atom probe microscope (experiment) is different compared to a large- - scale facility or electron accelerator experiments in at least two ways: - - * First, ionized atoms and molecular ion(s fragments) - (in the case of atom probe tomography) - and (primarily) imaging gas ions (in the case of field ion - microscopy) are accelerated towards a position-sensitive - and time-of-flight taking detector system. - Hence, there is no real probe/beam. - * Second, the specimen is the lens of the microscope. - - - - - Given name of the atom probe at the hosting institution. This is an - alias. Examples could be LEAP5000, Raptor, Oxcart, one atom at a time, - etc. - - - - - Location of the lab or place where the instrument is installed. - Using GEOREF is preferred. - - - - - - - - - - - The space inside the atom probe along which ions pass nominally - when they leave the specimen and travel to the detector. - - THIS DOCSTRING NEEDS CLARIFICATION. - - - - - - The nominal diameter of the specimen ROI which is measured in the - experiment. It is important to mention that the physical specimen - cannot be measured completely because ions may launch but not be - detected or hit elsewhere in the analysis_chamber. - - - - - - - Is a reflectron installed and was it used? - - - - - - - - - - - - - - - - A local electrode guiding the ion flight path. Also called - counter or extraction electrode. - - - - Identifier of the local_electrode in an e.g. database. - - - - - - - - - - - - - - - - Detector for taking raw time-of-flight and - ion/hit impact positions data. - - - - Description of the detector type. Specify if the detector is - not the usual type, i.e. not a delay-line detector. - In the case the detector is a multi-channel plate/ - delay line detector, use mcp_dld. In the case the detector is - a phosphor CCD use phosphor_ccd. In other case specify - the detector type via free-text. - - - - - - Given name/alias. - - - - - - Given brand or model name by the manufacturer. - - - - - Given hardware name/serial number or hash identifier - issued by the manufacturer. - - - - - Given name of the manufacturer. - - - - - Amplitude of the signal detected on the multi-channel plate (MCP). - - This field should be used for storing the signal amplitude quantity - within ATO files. The ATO file format is used primarily by the - atom probe groups of the GPM in Rouen, France. - - - - - - - - - - - - - - - - - - - Atom probe microscopes use controlled laser, voltage, or a - combination of pulsing strategies to trigger the excitation - and eventual field evaporation/emission of an ion during - an experiment. - If pulse_mode is set to laser or laser_and_voltage (e.g. for - LEAP6000-type instruments) having the group/section laser_gun - is required and the following of its fields have to be filled: - - * name - * wavelength - * energy - - - - - - - - - - - - - - - - - - - - - - Average temperature at the specimen base, i.e. - base_temperature during the measurement. - - - - - The best estimate, at experiment time, for the temperature at the - sample base (furthest point along sample apex and holding assembly - that is removable from the sample stage). - - - - - - - - - - - - - - - - - - - - Average pressure in the analysis chamber. - - - - - - - - - - - - - - - - Average pressure in the buffer chamber. - - - - - - - - - - - - - - - - Average pressure in the load_lock_chamber. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A possible place, which has to be discussed with the atom probe - community more though, where quantitative details about the calibration - of the counter electrode could be stored. Work in this direction was - e.g. reported by the `Erlangen group <https://www.youtube.com/watch?v=99hNEkqdj78t=1876s>`_ - (see `P. Felfer et al. <http://dx.doi.org/10.1016/j.ultramic.2016.07.008>`_ ) - - - - - - - A place where details about the initial shape of the specimen - can be stored. Ideally, here also data about the shape evolution - of the specimen can be stored. There are currently very few - techniques which can measure the shape evolution: - - * Correlative electron microscopy coupled with modeling - but this usually takes an interrupted experiment - in which the specimen is transferred, an image taken, - and a new evaporation sequence initiated. - Examples are `I. Mouton et al. <https://doi.org/10.1017/S1431927618016161>`_ - and `C. Fletcher <https://doi.org/10.1088/1361-6463/abaaa6>`_. - * Another method, which is less accurate though, is to monitor - the specimen evolution via the in-built camera system - (if available) in the instrument. - * Another method is to use correlated scanning force microscopy - methods like reported in `C. Fleischmann <https://doi.org/10.1016/j.ultramic.2018.08.010>`_. - * A continuous monitoring of the specimen in a - correlative electron microscopy/atom probe experiment - is planned to be developed by `T. Kelly et al. <https://doi.org/10.1017/S1431927620022205>`_ - Nothing can be said about the outcome of this research yet but - here is where such spatio-temporally data could be stored. - - - - - Ideally measured or best elaborated guess of the - initial radius of the specimen. - - - - - Ideally measured or best elaborated guess of the shank angle. - This is a measure of the specimen taper. Define it in such a way - that the base of the specimen is modelled as a conical frustrum so - that the shank angle is the (shortest) angle between the specimen - space z-axis and a vector on the lateral surface of the cone. - - - - - Average detection rate over the course of the experiment. - - - - - - Estimated field at the apex along the evaporation sequence. - - - - - - - - - The majority of atom probe microscopes come from a - single commercial manufacturer `AMETEK (formerly Cameca) <https://www.atomprobe.com>`_. - Their instruments are controlled via an(/a set) of integrated - instrument control system(s) (APSuite/IVAS/DAVis). - - By contrast, instruments which were built by individual - research groups such as of the French (GPM, Rouen, France), - the Schmitz (Inspico, Stuttgart, Germany), - the Felfer (Oxcart, Erlangen, Germany), - the Northwestern (D. Isheim, Seidman group et al.), - or the PNNL group (Pacific Northwest National Laborary, - Portland, Oregon, U.S.) have other solutions - to control the instrument. - - Some of which are modularized and open, - some of which realize also integrated control units with - portions of eventually undisclosed source code and - (so far) lacking (support of)/open APIs. - - Currently, there is no accepted/implemented - community-specific API for getting finely granularized - access to such control settings. - - These considerations motivated the design of the NXapm - application definition in that it stores quantities in NXcollection. - groups to begin with. Holding heterogeneous, not yet standardized - but relevant pieces of information is the purpose of this collection. - - - - - - - - - - Track time-dependent details over the course of the measurement about the - buffer_chamber. - - - - - Track time-dependent details over the course of the measurement about the - load_lock_chamber. - - - - - Track time-dependent details over the course of the measurement about the - analysis_chamber. - - - - - - - - A statement whether the measurement was successful or failed prematurely. - - - - - - - - - - - - - Details about where ions hit the ion_detector and data processing - steps related to analog-to-digital conversion of detector signals - into ion hit positions. For AMETEK LEAP instruments this processing - takes place partly in the control unit of the detector partly - in the software. The process is controlled by the acquisition/ - instrument control software (IVAS/APSuite/DAVis). - The exact details are not documented by AMETEK in an open manner. - For instruments built by individual research groups, - like the Oxcart instrument, individual timing data from the - delay-line detector are openly accessible. - - - - - - - - - - - Raw readings from the analog-to-digital-converter - timing circuits of the detector wires. - - - - - - - - - - Evaluated ion impact coordinates at the detector - (either as computed from the arrival time data - or as reported by the control software). - If the acquisition software enables it one can also store in this - field the hit_positions, as measured by the detector, without any - corrections. - - - - - - - - - - - This could be a place where currently the publicly undocumented - algorithmic steps are stored how detected hits are judged for their - quality. In CamecaRoot this there is something mentioned like - golden and partial hits, here is where this could be documented. - - - - - - - Data post-processing step which is, like the impact position analyses, - usually executed in the integrated control software. This processing - yields how many ions were detected with each pulse. - - It is possible that multiple ions evaporate and hit the same or - different pixels of the detector on the same pulse. - These data form the basis to analyses of the so-called - (hit) multiplicity of an ion. - - Multiplicity must not be confused with how many atoms - f the same element or isotope, respectively, a molecular - ion contains (which is instead encoded with the - isotope_vector field of each NXion instance). - - - - - - - - - - Number of pulses since the last detected ion pulse. - For multi-hit records, after the first record, this is zero. - - - - - - - - - Number of pulses since the start of the atom probe - run/evaporation sequence. - - - - - - - - - Hit multiplicity. - - - - - - - - - Like impact position and hit multiplicity computations, - ion filtering is a data post-processing step with which users - identify which of the detected ions should be included - in the voltage-and-bowl correction. - This post-processing is usually performed via GUI interaction - in the reconstruction pipeline of IVAS/APSuite. - - - - - - - - - - Bitmask which is set to true if the ion - is considered and false otherwise. - - - - - - - - - - Data post-processing step to correct for ion impact - position flight path differences, detector biases, - and nonlinearities. This step is usually performed - with commercial software. - - - - - - - - - - - Raw time-of-flight data as read out from the acquisition software - if these data are available and accessible. - - - - - - - - - Calibrated time-of-flight. - - - - - - - - The key idea and algorithm of the voltage-and-bowl correction is - qualitatively similar for instruments of different manufacturers - or research groups. - - Specific differences exists though in the form of different - calibration models. For now we do not wish to resolve or - generalize these differences. Rather the purpose of this collection - is to provide a container where model-specific parameters - and calibration models can be stored if users know these - for sure. - - For AMETEK LEAP instruments this should be the place for - storing initial calibration values. These values are - accessible normally only by AMETEK service engineers. - They use these for calibrating the detector and instrument. - - Users can also use this NXcollection for storing the - iteratively identified calibrations which scientists - will see displayed in e.g. APSuite while they execute - the voltage-and-bowl correction as a part of the - reconstruction pipeline in APSuite. - - - - - - - Data post-processing step in which calibrated time-of-flight data - (ToF) are interpreted into mass-to-charge-state ratios. - - - - - - - - - - Store vendor-specific calibration models here (if available). - - - - - Mass-to-charge-state ratio values. - - - - - - - - - - - Data post-processing step to create a tomographic reconstruction - of the specimen based on selected calibrated ion hit positions, - the evaporation sequence, and voltage curve data. - Very often scientists use own software scripts according to - published procedures, so-called reconstruction protocols, - i.e. numerical recipes how to compute x, y, z atomic positions - from the input data. - - - - - - - - - - Qualitative statement about which reconstruction protocol was used. - - - - - - - - - - - - - Different reconstruction protocols exist. Although these approaches - are qualitatively similar, each protocol uses different parameters - (and interprets these differently). The source code to IVAS/APSuite - is not open. For now users should store reconstruction parameter - in a collection. - - - - - - Different strategies for crystallographic calibration of the - reconstruction are possible. The field is required and details - should be specified in free-text at least. If the not crystallographic - calibration was performed the field should be filled with the n/a, - meaning not applied. - - - - - Three-dimensional reconstructed positions of the ions. - Interleaved array of x, y, z positions in the specimen space. - - - - - - - - - - An array of triplets of integers which can serve as a supplementary - array for Paraview to display the reconstructed dataset. - The XDMF primitive type is here 1, the number of primitives 1 per - triplet, the last integer in each triplet is the identifier of - each point starting from zero. - - - - - - - - - - Six equally formatted sextets chained together. For each - sextett the first entry is an XDMF primitive topology - key (here 5 for polygon), the second entry the XDMF primitive - count value (here 4 because each face is a quad). - The remaining four values are the vertex indices. - - - - - - - - To get a first overview of the reconstructed dataset, - the format conversion computes a simple 3d histogram - of the ion density using one nanometer cubic bins without - applying smoothening algorithms on this histogram. - - - - - - - - - A default three-dimensional histogram of the total - number of ions in each bin obtained via using a rectangular - transfer function. - - - - - - - - - Array of counts for each bin. - - - - - - - - - - Bin center of mass position along the z axis. - - - - - - - - - Bin center of mass position along the y axis. - - - - - - - - - Bin center of mass position along the x axis. - - - - - - - - - - - - - Data post-processing step in which elemental, isotopic, - and/or molecular identities are assigned to the ions. - The documentation of these steps is based on ideas - described in the literature: - - * `M. K. Miller <https://doi.org/10.1002/sia.1719>`_ - * `D. Haley et al. <https://doi.org/10.1017/S1431927620024290>`_ - * `M. Kühbach et al. <https://doi.org/10.1017/S1431927621012241>`_ - - - - - - - - - - How many ion types are distinguished. - If no ranging was performed each ion is of the special unknown type. - The iontype ID of this unknown type is 0 which is a reserve value. - Consequently, iontypes start counting from 1. - - - - - Assumed maximum value that suffices to store all relevant - molecular ions, even the most complicated ones. - Currently a value of 32 is used. - - - - - Specifies the computation of the mass-to-charge histogram. - Usually mass-to-charge values are studied as an ensemble quantity, - specifically these values are binned. - This (NXprocess) stores the settings of this binning. - - - - - - - - - Smallest and largest mass-to-charge-state ratio value. - - - - - - - - - Binning width of the mass-to-charge-state ratio values. - - - - - - A default histogram aka mass spectrum of - the mass-to-charge-state ratio values. - - - - - - - - - Array of counts for each bin. - - - - - - - - - Right boundary of each mass-to-charge-state ratio value bin. - - - - - - - - - - - - Details of the background model which was used to - correct the total counts per bin into counts. - - - - - - - - - - - How where peaks in the background-corrected in the histogram - of mass-to-charge-state ratio values identified? - - - - - - - - - - - THIS DOCSTRING NEEDS CLARIFICATION. - - - - - - - Details about how peaks, with taking into account - error models, were interpreted as ion types or not. - - - - - - - - - - - - - - - - - - diff --git a/applications/NXem.nxdl.xml b/applications/NXem.nxdl.xml deleted file mode 100644 index 0509b90861..0000000000 --- a/applications/NXem.nxdl.xml +++ /dev/null @@ -1,2036 +0,0 @@ - - - - - - - Characterization of a sample during a session on an electron microscope. - - **The idea and aim of NXem**: - Electron microscopy (EM) research, whether it be performed with scanning electron - microscope (SEM) or transmission electron microscope (TEM) instruments, uses - versatile tools for preparing and characterizing samples and specimens. - The term specimen is considered a synonym for sample in this application definition. - A specimen is a physical portion of material that is studied/characterized during - the microscope session, eventually in different places on the specimen surface, - illuminating the surface layers or shining through thin specimens. - These places are named regions of interest (ROIs). - - Fundamentally, an electron microscope is an electron accelerator. - Experimentalists use it in sessions during which they characterize as well as - prepare specimens. This application definition describes data and metadata about - processes and characterization tasks applied to one specimen. - The application definition focuses on the usage of EM in materials research. - The application definition design makes it in principle applicable also in - cryo-EM on biomaterials. - - Multiple specimens have to be described with multiple :ref:`NXentry` instances. - - **Electron microscopes motivate the development of a comprehensive data schema:** - There are research groups who use an EM in a manner where it is exclusively - operated by a single, instrument-responsible scientists or a team of scientists. - These users may perform analyses for other users as a service task, especially - in large research facility settings. Oftentimes, though, and especially for - cutting-edge instruments, the scientists guide the process and maybe even control - the microscope. Instruments are usually controlled on-premises but also more - and more functionalities for remote control have become available. - Scientists oftentimes can ask technicians for support. In all cases, - these people are considered users. Users might have different - roles though. - - The rational behind a common EM schema rather than making separate schemas - for SEM or TEM are primarily the key similarities of SEM and TEM - instruments: - - Both type of instruments have electro-magnetic lenses. These may differ in - design, alignment, number, and level of corrected for aberrations. - As an obvious difference, a TEM is mainly used for measuring the transmitted - electron beam. This calls for using a different lens setup and relative - placement of the specimen in the lens setup. Also TEM specimens are - substantially thinner than specimens characterized with SEM to enable an - illumination through the specimen. This offers capabilities for probing of - additional physical mechanisms of electron-matter interaction which are - unavailable in SEMs. - - Nevertheless, both types of electron microscopes use detector systems which - measure different types of signals that originate from the same set of - radiation/specimen interactions. Often these detectors have a similar design - and technology or are even used both in SEMs and TEMs. - - **A comprehensive schema instead of specific SEM or TEM schemas**: - Given these physical and technical differences, different instruments have - been developed. This led to a coexistence of two broad interacting - communities: SEM and TEM users. From a data science perspective, we - acknowledge that the more specific a research question is and the narrower it - is the addressed circle of users which develops or uses schemas for research - data management (RDM) with EM, the more understandable it is that scientists - of either community (or sub-community) ask for designing method-specific schemas. - - Researchers who have a single (main) microscope of some vendor in their lab, - may argue they need an NXem_vendor_name schema or an NXem_microscope_name or - an NXem_sem or a NXem_tem schema. - Scientists exclusively working with one technique or type of signal probed - (X-rays, electrons) may argue they wish to be pragmatic and store only - what is immediately relevant for their particular technique and - research questions. In effect, they may advocate for method-specific - schemas such as NXem_ebsd, NXem_eels, NXem_edx, or NXem_imaging. - - However, the history of electron microscopy has shown that these activities led - to a zoo of schemas and vocabulary, with implementation in many data and file formats, - difficult to make interoperable. Instead of trying to maintain this, we would like - to advocate that the `FAIR principles <https://doi.org/10.1038/sdata.2016.18>`_ - should guide all decisions how data and metadata should be stored. - - EM instruments, software, and research are moving targets. Consequently, - there is a key challenge and inconvenience with having many different schemas - with associated representations of data and metadata: Each combination of - schemas or an interoperable-made handshake between two file formats or software - packages has to be maintained by software developers. This counts especially - when data should be processed interoperably between software packages. - - This brings two problems: Many software tools and parsers for the handshaking - between tools have to be maintained. This can result in usage of different - terminology, which in turn results in representations and connections made - between different data representations and workflows that are not - machine-actionable. - `There are community efforts to harmonize the terminology. <https://gitlab.hzdr.de/em_glossary/em_glossary>`_ - - **The advantage of working towards a common vocabulary and representation**: - A common vocabulary can serve interoperability as developers of schemas and - scientists can reuse for instance these terms, thus supporting interoperability. - Ideally, scientists specialize an application definition only for the few very - specific additional quantities of their instruments and techniques. This is - better than reimplementing the wheel for descriptions of EM instruments. - This route of more standardization can support the EM community in that it - removes the necessity for having to maintain a very large number of schemas. - - Aiming for more standardization, i.e. a lower number of schemas rather than - a single standard for electron microscopy is a compromise that can serve - academia and industry as it enables a focusing of software development - efforts on those schemas, on fixing and discussing them, and on harmonizing - their common vocabulary. These activities can be specifically relevant also - for technology partners building EM hard- and software as it improves the - longevity of certain schemas; and thus can help with incentivizing them to support - the community with implementing support for such schemas into their applications. - - In effect, everybody can gain from this as it will likely reduce the cases - in which scientists have to fix bugs in making their own tools compliant and - interoperable with tools of their colleagues and the wider community. - - The here proposed NXem application definition offers modular components - (EM-research specific base classes) for defining schemas for EM research. - Thereby, NXem can be useful to extends top-level ontologies towards a domain- - and method-specific ontology for electron microscopy as it is used for - materials research. - - Working towards a common vocabulary is a community activity that profits from - everybody reflecting in detail whether certain terms they have used in the past - are not eventually conceptually similar if not the same as what this application - definition and its base classes provide. We are happy for receiving your feedback. - - **Addressing the generality versus specificity challenge**: - It is noteworthy to understand that (not only for NeXus), schemas differ - already if at least one field is required in one version of the schema, - but it is set optional in another schema. If group(s), field(s), or - attributes are removed or added, or even a docstring is changed, schemas can - become inconsistent. It is noteworthy to mention that the idea of a NeXus application - definition serves as a contract between a data provider and a data consumer. - Providers can be the software of a specific microscopes or users with specific - analysis needs. Consumers can be again specific software tools, like vendor - software for controlling the instrument or a scientific software for doing - artificial intelligence analyses on EM data). Such changes of a schema lead - to new versions. - - **Verification of constraints and conditions**: - Tools like NeXus do not avoid or protect against all such possible inconsistencies; - however NeXus offers a mechanism and toolset, through which schemas can be - documented and defined. In effect, having an openly documented - (at a case-specific level of technical detail) schema is a necessary but alone - not a sufficient step to take EM research on a route of machine-actionable - and interoperable FAIR data. - - This stresses again the fundamental and necessary role of working towards - a common vocabulary and, with a longer perspective in mind, a machine-actionable - knowledge representation and verification engine. So far many conditions and - requirements are formulated in the docstrings of the respective entries of - the application definition. - - **NXem takes a key step towards standardization of EM data schemas**. - It offers a controlled vocabulary and set of relations between concepts and - enables the description of the data which are collected for research with - electron microscopes. To be most efficient and offering reusability, the NXem - application definition should be understood as a template that one should - ideally use as is. NXem can be considered a base for designing more specialized - definitions. These should ideally be prefixed with NXem_method (e.g. NXem_ebsd). - - **The use of NXem should be as follows:** - Offspring application definitions should not remove groups but leave these - optional or, even better, propose changes to NXem. - - A particular challenge with electron microscopes as physical instruments are - their dynamics. To make EM data understandable, repeatable, and eventually - corresponding experiments reproducible in general requires a documentation - of the spatio-temporal dynamics of the instrument in its environment. - It is questionable to which level such a reproducibility is possible with EM - at all considering beam damage, effects of the environment, and other not - exactly quantifiable influences. - While this points to the physical limitations there are also practical and - economical constraints on how completely EM research can be documented: - For most commercial systems there is a specific accessibility beyond which - detailed settings like lens excitations and low-level hardware settings - may not be retrievable as technology partners have a substantiated interest in - finding a compromise between being open to their users and securing their - business models. - - By design, EM experiments illuminate the specimen with electrons as a - consequence of which the specimen changes if not may get destroyed. - As such, repeatability of numerical processing and clear descriptions of - procedures and system setups should be addressed first. - - If especially a certain simulation package needs a detailed view of the - geometry of the lens system and its excitations during the course of the - experiment, it is difficult to fully abstract the technical details of the - hardware into a set of names for fields and groups that make for a compromise - between clarity but being system-agnostic at the same time. - Settings of apertures are an example where aperture modes are in most cases - aliases behind which there is a set of very detailed settings specific to the - software and control units used. These settings are difficult to retrieve, - are not fully documented by technology partners. This simplification for - users of microscopes makes experiments easier understandable. - On the flipside these subtilities limit the opportunities of especially open- - source developments to make data schemas covering enough for general usage and - specific enough and sufficiently detailed to remain useful for - research by electron microscopy domain experts. - - Instead, currently it is for the docstring to specify what is conceptually - eventually behind such aliases. The design rule we followed while drafting - this NXem application definition and base classes is that there are numerous - (technical) details about an EM which may warrant a very detailed technical - disentangling of settings and reflection of numerous settings as deeply - nested groups, fields and attributes. An application definition can offer a - place to hold these nested representations; however as discussed - at the cost of generality. - - Which specific details matter for answering scientific research questions is - a difficult question to answer by a single team of scientists, especially - if the application definition is to speak for a number of vendors. What makes - it especially challenging is when the application definition is expected to - hold all data that might be of relevance for future questions. - - We are skeptical if there is one such representation that can fulfill all these - aims and interest, while remaining at the same time approachable and executable - by a large number of scientists in a community. However, we are also convinced - that this is not a reason to accept the status quo of having a very large set - of oftentimes strongly overlapping and redundant schemas. - - NXem is our proposal to motivate the EM community to work towards more - standardization and discussion of what constitutes data, i.e. metadata, - numerical and categorical data in research with electron microscopes. We found - that existent terminology can be encoded into a more controlled vocabulary. - - We have concluded that despite all these details of current EM research with - SEM, TEM, and focused-ion beam instruments, there a clearly identifiable - common components and generalizable settings of EM research use cases. - Therefore, - - **This application definition has the following components at the top-level:** - - * Each signal, such as a spectrum or image taken at the microscope, should - have an associated time-zone-aware time stamp and report of the specific - settings of the microscope at that point in time when the image was taken. - This is why instances of :ref:`NXevent_data_em` have their own em_lab section. - The reason is that EMs can be highly dynamic, used to illuminate the specimen - differently or show drift during signal acquisition, to name but a few effects. - What constitutes a single EM experiment/measurement? - This can be the collecting of a single diffraction pattern with a scanning TEM (STEM), - taking of a secondary electron image for fracture analysis, taking a set of - EBSD line scans and/or surface mappings in an SEM, or the ion-beam-milling of a - specimen in preparation for e.g. an atom probe experiment. - * :ref:`NXmonitor`; - instances to keep track of time-dependent quantities - pertaining to specific components of the instrument. - Alternatively, NXevent_data_em instances can be used to store - time-zone-aware dates of the components, which is - relevant for documenting as exactly as practically possible settings - when images and spectra were taken. - * :ref:`NXinstrument`; - conceptually this is a container to store an arbitrary level of detail of the - technical components of the microscope as a device and the lab in which - the microscope is operated. - * :ref:`NXuser`; - conceptually, this is a set with at least one NXuser instance which details - who operated or performed the measurement. Additional NXusers can be - referred to in an NXevent_data_em instance to store - individualized details of who executed an event of data acquisition or processing. - * :ref:`NXevent_data_em` instances as an NXevent_data_em_set; - each NXevent_data_em instance is a container to group specific details - about the state of the microscope when a measurement was taken and - relevant data and eventual processing steps were taken (on-the-fly). - * :ref:`NXdata`; at the top-level, this is a place for documenting available - default plottable data. A default plottable can be useful for research data - management systems to show a visual representation of some - aspect of the content of the EM session. - Default plottables are not intended to serve every possible analysis and - visualization demand but are instead a preview. We made this choice - because what constitutes a useful default plot is often a matter of interpretation, - somewhat of personal taste, and community standards. In effect, default - plottables are case- and method-specific. - - Usually a session at a microscope is used to collect multiple signals. - Examples for possible default plottables could be arbitrarily taken secondary, - back-scattered, electron image, diffraction pattern, EELS spectra, composition, - or orientation mappings to name but a few. - - **There are a few design choices to consider with sub-ordinate groups:** - - * Images and spectra should be stored as :ref:`NXimage_set` and :ref:`NXspectrum_set` - instances to hold data at the earliest possible step in the computational - processing (which is usually performed with the microscope control and or - integrated analysis software). The formatting of the NXdata groups enables the - display of image and spectra with web technology visualization software. - * When two- and three-dimensional geometric primitive data are stored, it is useful - to write additional optional `XDMF <https://www.xdmf.org/index.php/XDMF_Model_and_Format>`_ - fields which support additional plotting of the data with visualization software. - * Consumable results of EM characterization tasks are usually a sub-set of - data artifacts, as there is not an infinite amount of possible - electron/ion beam-specimen interactions. - * Images based on electron counts are typically detected with specific operation modes - such as bright field or dark field imaging in TEM or secondary/back-scattered electron - imaging in SEM. - * Also spectra (X-ray quanta or Auger electron counts) typically are referred to - under the assumption of a specific operation mode of the microscope. - * These data are in virtually all cases a result of some numerical processing. - These data and processing steps are modelled as instances of :ref:`NXprocess` - which use terms from a controlled vocabulary e.g. SE (secondary electron), - BSE (back-scattered electron), Kikuchi, X-ray, Auger, Cathodolum(inescence). - - **A key question often asked with EM experiments is how the actual (meta)data - should be stored (in memory or on disk)**. - - The application definition NXem is a graph which describes how numerical data - and (meta)data for EM research are related to one another. - - Electron microscopy experiments are usually controlled/performed via - commercial integrated acquisition and instrument control software. - In many cases, an EM dataset is useful only if it gets post-processed - already during the acquisition, i.e. while the scientist is sitting - at the microscope. - Many of these processes are automated, while some demand GUI - interactions with the control software. Examples include collecting - of diffraction pattern and on-the-fly indexing of these. - - It is possible that different types of programs might be used to - perform these processing steps whether on-the-fly or not. If this is - the case the processing should be structured with individual :ref:`NXprocess` - instances. If the program and/or version used for processing referred - to in an NXprocess group is different to the program and version - mentioned in this field, the NXprocess needs - to hold an own program and version. - - - - - - An at least as strong as SHA256 hashvalue of the file - that specifies the application definition. - - - - - - NeXus NXDL schema to which this file conforms. - - - - - - - - Ideally, a (globally) unique persistent identifier - for referring to this experiment. - - The identifier is usually defined/issued by the facility, - laboratory, or the principle investigator. - The identifier enables to link experiments to e.g. proposals. - - - - - Free-text description about the experiment. - - Users are strongly advised to detail the sample history in the respective - field and fill rather as completely as possible the fields of this - application definition rather than write details about the experiment - into this free-text description field. - - - - - ISO 8601 time code with local time zone offset to UTC information included - when the microscope session started. If the application demands that time - codes in this section of the application definition should only be used - for specifying when the experiment was performed - and the exact - duration is not relevant - this start_time field should be used. - - Often though it is useful to specify a time interval by specifying both - a start_time and an end_time to allow for more detailed bookkeeping and - interpretation of the experiment. The user should be aware that even - with having both time instances specified, it may not be possible - to infer how long the experiment took or for how long data were acquired. - - More detailed timing data over the course of the experiment have - to be collected to compute this. These computations can take - advantage of individual time stamps in NXevent_data_em instances to - provide additional pieces of information. - - - - - ISO 8601 time code with local time zone offset to UTC included when - the microscope session ended. - - - - - - - - - - Binary container for a file or a compressed collection of files which - can be used to add further descriptions and details to the experiment. - The container can hold a compressed archive. - - - - - A small image that is representative of the entry; this can be an - image taken from the dataset like a thumbnail of a spectrum. - A 640 x 480 pixel jpeg image is recommended. - Adding a scale bar to that image is recommended but not required - as the main purpose of the thumbnail is to provide e.g. thumbnail - images for displaying them in data repositories. - - - - - - Contact information and eventually details of at least one person - involved in the taking of the microscope session. This can be the - principle investigator who performed this experiment. - Adding multiple users if relevant is recommended. - - - - Given (first) name and surname of the user. - - - - - Name of the affiliation of the user at the point in time - when the experiment was performed. - - - - - Postal address of the affiliation. - - - - - Email address of the user at the point in time when the experiment - was performed. Writing the most permanently used email is recommended. - - - - - Globally unique identifier of the user as offered by services - like ORCID or ResearcherID. If this field is field the specific service - should also be written in orcid_platform - - - - - Name of the OrcID or ResearcherID where the account - under orcid is registered. - - - - - (Business) (tele)phone number of the user at the point - in time when the experiment was performed. - - - - - Which role does the user have in the place and at the point - in time when the experiment was performed? Technician operating - the microscope. Student, postdoc, principle investigator, guest - are common examples. - - - - - Account name that is associated with the user in social media platforms. - - - - - Name of the social media platform where the account - under social_media_name is registered. - - - - - - - A description of the material characterized in the experiment. - Sample and specimen are threaded as de facto synonyms. - - - - A qualifier whether the sample is a real one or a - virtual one (in a computer simulation) - - - - - - - - - - Ideally (globally) unique persistent identifier. The name distinguishes - the specimen from all others and especially the predecessor/origin - from where the specimen was cut. - - This field must not be used for an alias of the sample. - Instead, use short_title for this, more convenient alias name. - - In cases where multiple specimens have been loaded into the microscope - the name has to identify the specific one, whose results are stored - by this NXentry, because a single NXentry should be used only for - the characterization of a single specimen. - - Details about the specimen preparation should be stored in the - sample history. - - - - - Ideally, a reference to a (globally) unique persistent identifier, - representing a data artifact which documents ideally as many details - of the material, its microstructure, and its thermo-chemo-mechanical - processing/preparation history as possible. - - The sample_history is the record what happened before the specimen - was placed into the microscope at the beginning of the session. - - In the case that such a detailed history of the sample/specimen is not - available, use this field as a free-text description to specify a - sub-set of the entire sample history, i.e. what you would consider are - the key steps and relevant information about the specimen, - its material, microstructure, thermo-chemo-mechanical processing state, - and the details of the preparation. - - Specific details about eventual physically-connected material like - embedding resin should be documented ideally also in the sample_history. - If all fails, the description field can be used but it is strongly - discouraged because it leads to eventually non-machine-actionable - data. - - - - - ISO 8601 time code with local time zone offset to UTC information - when the specimen was prepared. - - Ideally report the end of the preparation, i.e. the last known time - the measured specimen surface was actively prepared. Usually this should - be a part of the sample history, i.e. the sample is imagined handed over - for analysis. - - Knowing when the specimen was exposed to e.g. specific atmosphere is - especially required for environmentally sensitive material such as - hydrogen charged specimens or experiments - including tracers with a short half time. Further time stamps prior - to preparation_date should better be placed in resources which - describe the sample_history. - - - - - Possibility to give an abbreviation or alias of the specimen name field. - - - - - List of comma-separated elements from the periodic table that are - contained in the sample. If the sample substance has multiple - components, all elements from each component must be included in - `atom_types`. - - The purpose of the field is to offer materials database systems an - opportunity to parse the relevant elements without having to interpret - these from the sample history. - - - - - - (Measured) sample thickness. The information is recorded to qualify - if the beam used was likely able to shine through the specimen. - For scanning electron microscopy, in many cases the specimen is much - thicker than what is illuminatable by the electron beam. - In this case the value should be set to the actual thickness of - the specimen viewed for an illumination situation where the nominal - surface normal of the specimen is parallel to the optical axis. - - - - - - (Measured) density of the specimen. For multi-layered specimens this - field should only be used to describe the density of the excited - volume. For scanning electron microscopy the usage of this field is - discouraged and instead an instance of an :ref:`NXinteraction_vol_em` - within individual :ref:`NXevent_data_em` instances can provide a much - better description of the relevant details why one may wish to store - the density of the specimen. - - - - - - Discouraged free-text field in case properly designed records - for the sample_history are not available. - - - - - - Hard link to a location in the hierarchy of the NeXus file - where the data for default plotting are stored. - - - - - - - - - - - - Metadata and numerical data of the microscope and the lab in which it stands. - - The em_lab section contains a description of the instrument and its components. - The component descriptions in this section differ from those inside individual - NXevent_data_em sections. These event instances take the role of time snapshot. - For an NXevent_data_em instance users should store only those settings for a - component which are relevant to understand the current state of the component. - Here, current means at the point in time, i.e. the time interval, - which the event represents. - - For example it is not relevant to store in each event's electron_source - group again the details of the gun type and manufacturer but only the - high-voltage if for that event the high-voltage was different. If for all - events the high-voltage was the same it is not even necessary to include - an electron_source section in the event. - - Individual sections of specific type should have the following names: - - * NXaperture: the name should match with the name of the lens - * NXlens_em: condenser_lens, objective_lens are commonly used names - * NXcorrector_cs: device for correcting spherical aberrations - * NXstage_lab: a collection of component for holding the specimen and - eventual additional component for applying external stimuli on the sample - * NXdetector: several possible names like secondary_electron, - backscattered_electron, direct_electron, ebsd, edx, wds, auger, - cathodoluminescence, camera, ronchigram - - - - Given name of the microscope at the hosting institution. This is an alias. - Examples could be NionHermes, Titan, JEOL, Gemini, etc. - - - - - Location of the lab or place where the instrument is installed. - Using GEOREF is preferred. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If the lens is described at least one of the fields - voltage, current, or value should be defined. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If the lens is described at least one of the fields - voltage, current, or value should be defined. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Description of the type of the detector. - - Electron microscopes have typically multiple detectors. - Different technologies are in use like CCD, scintillator, - direct electron, CMOS, or image plate to name but a few. - - - - Instrument-specific alias/name - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A container for storing a set of NXevent_data_em instances. - - An event is a time interval during which the microscope was configured - in a specific way. The microscope is considered as stable enough during - this interval that a measurement with one or multiple detectors is - possible. What constitutes such time interval depends on how the - microscope is used and which measurements the user would like to perform. - - Each NXevent_data_em instance holds one acquisition task with detectors. - Each NXevent_data_em section contains an em_lab group in which specific - settings and states of the microscope during the time interval - can be stored to document the history of states of the microscope over - the course of the session. - - The NXem application definition offers maximal flexibility. - One extreme could be that the only one NXevent_data_em instance is used - and this covers the time interval of the entire session at the microscope. - The other extreme case is that each collection of an image or even - single spot measurement is defined as an NXevent_data_em instance. - In this case the em_lab group inside the NXevent_data_em also holds - the specific time-dependent state of the microscope with which in theory - all dynamics of the system (if measured) can be captured and documented. - - Nowadays microscopes exists for which hard- and software solutions - enable a tracking of the dynamics of the microscope and the actions - of the user (such as with solution like AXONSynchronicity from Protochips). - The NXem application definition can however also be used for less - complex interaction and lower demands wrt to time tracking activities. - - An NXevent_data_em instance holds specific details about how raw data from - a detector were processed. Raw data may already be post-processed as - they are accessible only by the control software with after some internal - processing happened. Nevertheless, these data have to be distinguished from - post-processed data where e.g. raw data are converted to interpreted - spectra, or orientation mappings. - - This post-processing tasks can be performed (on-the-fly, i.e. during - acquisition for sure during the microscope session) or afterwards. - Post-processing is performed with commercial software or various - types and scripts. - - Currently, several specializations of NXimage_set and Nspectrum_set - are used which store some details of this processing. However, as post- - processing tasks can be substantially more advanced and involved it - is clear that data artifacts from the measurement and data artifacts - generated during post-processing are weakly connected only, maybe - exclusively by the fact that a complex numerical post-processing workflow - just takes one raw dataset from an NXevent_data_em instance but generates - multiple derived data artifacts from this. All these should be described - as own application definitions and only weak connections should be made - to an instance of NXem. Instances of NXsubentry is one mechanism in - NeXus how this can be achieved in the future. - - - - A container holding a specific result of the measurement and - eventually metadata how that result was obtained numerically. - - NXevent_data_em instances can hold several specific NXimage_em or - NXspectrum_em instances taken and considered as one event, i.e. - a point in time when the microscope had the settings specified - either in NXinstrument or in this NXevent_data_em instance. - - The application definition is designed without an explicit need for - having an NXevent_data_em instance that contains an NXimage_em or - NXspectra_em instance. Thereby, an NXevent_data_em can also be used - for just documentation about the specific state of the microscope - irrespective whether data have been collected during this time interval. - - In other words the NXinstrument group details primarily the more - static settings and components of the microscope as they are found - by the operator during the session. The NXevent_data_em samples - the dynamics. - - It is not necessary to store data in NXebeam, NXibeam instances - of NXevent_data_em but in this case it is assumed that the settings - were constant over the entire course of the microscope session - and thus all relevant metadata inside the NXinstrument groups - are sufficient to understand the session. - - So far there exists no standard which a majority of the technology - partners and the materials science electron microscopy community - have accepted which could be used for a very generic documentation, - storage and exchange of electron microscope data. Therefore, it is - still a frequent case that specific files have many fields which cannot - safely be mapped or interpreted. - **Therefore, users are always given the advice to keep the vendor files.** - Working however with these vendor files inside specific software, - like materials databases, demands for parsers which extract pieces of - information from the vendor representation (numerical data and metadata) - and map them on a schema with which the database and its associated - software tools can work with. - - Currently, one would loose immediately track of e.g. provenance and - the origin of certain data in NXevent_data_em instances unless really - all data are safely and reliably copied over into an instance of the - schema. Currently, though, this is sadly effectively prevented in many - cases as vendors indeed implemented often sophisticated provenance - and commercial software state tracking tools but these are not yet - documented covering enough in our opinion so that it is safe to assume - all vendor field names are known, logically understood, interpretable, - and thus mappable on a common schema using a controlled common - vocabulary. - - Therefore we encourage user for now to store for each NXimage_set - or NXspectra_set instance to supply the so-called source of the data. - This can for instance be the name and hashvalue of the original - file which was acquired during the microscope session and from which then - certain details like numerical data and metadata were copied into an - instance of this schema for the purpose of working with the data in - e.g. tools offered by research data management (RDM) systems or - materials database. - - During our work on implementing file format/metadata parsers and - developing this application definition, we realized that **several - software tools currently do not consistently format critical metadata - like time-zone-aware timestamps** when events of data collection or - processing happened. - - We would like to encourage the community and especially the vendors - to work towards a standardization, or at least an open documentation - of the way how time-zone-aware time data are collected and stored how - and where during a microscope session and how they end up in files - and databases with which users interact. - This would enable to supplement instances of this schema with specific - time data and assure that these time data can be used to reliably - contextualize individual datasets and processing steps in materials - information systems. - - For the reason that these measures have not yet been fully taken, - the start_time and end_time is a recommended option. - The idea behind these time-zone-aware dates is to identify when - the data were collected at the microscope but NOT when they were - transcoded by some software tool(s) while storing the data in an - instance of this schema. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Annulus inner half angle - - - - - Annulus outer half angle - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/base_classes/NXaberration.nxdl.xml b/base_classes/NXaberration.nxdl.xml deleted file mode 100644 index edfc744506..0000000000 --- a/base_classes/NXaberration.nxdl.xml +++ /dev/null @@ -1,55 +0,0 @@ - - - - - - Quantified aberration coefficient in an aberration_model. - - - - - Confidence - - - - - How was the uncertainty quantified e.g. via the 95% confidence interval. - - - - - Time elapsed since the last measurement. - - - - - For the CEOS definitions the C aberrations are radial-symmetric and have no - angle entry, while the A, B, D, S, or R aberrations are n-fold - symmetric and have an angle entry. - For the NION definitions the coordinate system differs to the one - used in CEOS and instead two aberration coefficients a and b are used. - - - - - diff --git a/base_classes/NXcg_alpha_complex.nxdl.xml b/base_classes/NXcg_alpha_complex.nxdl.xml index b6a466bc58..edf367ba06 100644 --- a/base_classes/NXcg_alpha_complex.nxdl.xml +++ b/base_classes/NXcg_alpha_complex.nxdl.xml @@ -2,9 +2,9 @@ - - + + The symbols used in the schema to specify e.g. dimensions of arrays. @@ -59,96 +60,90 @@ - Computational geometry description of geometric primitives via a face and edge list. + Computational geometry of primitives via a face-and-edge-list data structure. - Primitives must not be degenerated or self-intersect. - Such descriptions of primitives are frequently used for triangles and polyhedra - to store them on disk for visualization purposes. Although storage efficient, - such a description is not well suited for topological and neighborhood queries - of especially meshes that are built from primitives. + Primitives must neither be degenerated nor self-intersect but can differ in + their properties. A face-and-edge-list-based description of primitives is + frequently used for triangles and polyhedra to store them on disk for + visualization purposes (see OFF, PLY, VTK, or STL file formats). - In this case, scientists may need a different view on the primitives which - is better represented for instance with a half_edge_data_structure instance. - The reason to split thus the geometric description of primitives, sets, and - specifically meshes of primitives is to keep the structure simple enough for - users without these computational geometry demands but also enable those more - computational geometry savy users the storing of the additionally relevant - data structure. + Although this description is storage efficient it is not well suited for + topological analyses though. In this case, scientists may need a different + view on the primitives which is better represented with e.g. a + half_edge_data_structure. - This is beneficial and superior over NXoff_geometry because for instance a - half_edge_data_structure instance can be immediately use to reinstantiate - the set without having to recompute the half_edge_structure from the vertex - and face-list based representation and thus offer a more efficient route - to serve applications where topological and graph-based operations are key. + Having an own base class for the data structure how primitives are stored is + useful to embrace both users with small or very detailed specification demands. + + + Hint to help resolve in which Euclidean coordinate system + field values vertices are defined. + + - Dimensionality. + Dimensionality of the primitives described. - + - Array which specifies of how many vertices each face is built. - Each entry represent the total number of vertices for face, irrespectively - whether vertices are shared among faces/are unique or not. + Number of vertices for each face. + + Each entry represents the total number of vertices for that face, + irrespectively whether vertices are shared among faces or not. - + - Number of edges. + Number of edges for each face. + + Each entry represents the total number of edges for that face, + irrespectively whether edges are shared across faces or not. + + + - + - Number of faces. + Number of faces of the primitives. - Integer which specifies the first index to be used for distinguishing - identifiers for vertices. Identifiers are defined either implicitly - or explicitly. For implicit indexing the identifiers are defined on the - interval [identifier_offset, identifier_offset+c-1]. - For explicit indexing the identifier array has to be defined. + Integer offset whereby the identifier of the first member + of the vertices differs from zero. - The identifier_offset field can for example be used to communicate if - identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) - or from 0 (referred to as C-, Python-style index notation) respectively. + Identifier can be defined explicitly or implicitly. + Inspect the definition of NXcg_primitive_set for further details. - Integer which specifies the first index to be used for distinguishing - identifiers for edges. Identifiers are defined either implicitly - or explicitly. For implicit indexing the identifiers are defined on the - interval [identifier_offset, identifier_offset+c-1]. - For explicit indexing the identifier array has to be defined. + Integer offset whereby the identifier of the first member + of the edges differs from zero. - The identifier_offset field can for example be used to communicate if - identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) - or from 0 (referred to as C-, Python-style index notation) respectively. + Identifier can be defined explicitly or implicitly. + Inspect the definition of NXcg_primitive_set for further details. - Integer which specifies the first index to be used for distinguishing - identifiers for faces. Identifiers are defined either implicitly - or explicitly. For implicit indexing the identifiers are defined on the - interval [identifier_offset, identifier_offset+c-1]. - For explicit indexing the identifier array has to be defined. + Integer offset whereby the identifier of the first member + of the faces differs from zero. - The identifier_offset field can for example be used to communicate if - identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) - or from 0 (referred to as C-, Python-style index notation) respectively. + Identifier can be defined explicitly or implicitly. + Inspect the definition of NXcg_primitive_set for further details. - Integer used to distinguish vertices explicitly. + Integer identifier to distinguish all vertices explicitly. @@ -156,7 +151,7 @@ - Integer used to distinguish edges explicitly. + Integer used to distinguish all edges explicitly. @@ -164,23 +159,21 @@ - Integer used to distinguish faces explicitly. + Integer used to distinguish all faces explicitly. - + Positions of the vertices. - Users are encouraged to reduce the vertices to unique set of positions - and vertices as this supports a more efficient storage of the geometry data. + Users are encouraged to reduce the vertices to a unique set as this may + result in a more efficient storage of the geometry data. It is also possible though to store the vertex positions naively in which - case vertices_are_unique is likely False. - Naively here means that one for example stores each vertex of a triangle - mesh even though many vertices are shared between triangles and thus - the positions of these vertices do not have to be duplicated. + case vertices_are_unique is likely False. Naively here means that each + vertex is stored even though many share the same positions. @@ -189,7 +182,7 @@ - The edges are stored as a pairs of vertex identifier values. + The edges are stored as pairs of vertex identifier. @@ -198,7 +191,7 @@ - Array of identifiers from vertices which describe each face. + The faces are stored as a concatenated array of vertex identifier tuples. The first entry is the identifier of the start vertex of the first face, followed by the second vertex of the first face, until the last vertex @@ -207,7 +200,7 @@ Therefore, summating over the number_of_vertices, allows to extract the vertex identifiers for the i-th face on the following index interval - of the faces array: [$\sum_i = 0}^{i = n-1}$, $\sum_{i=0}^{i = n}$]. + of the faces array: :math:`[\sum_{i = 0}^{i = n-1}, \sum_{i=0}^{i = n}]`. @@ -216,18 +209,23 @@ - If true indicates that the vertices are all placed at different positions - and have different identifiers, i.e. no points overlap or are counted twice. + If true, indicates that the vertices are all placed at different positions + and have different identifiers, i.e. no points overlap or are counted more + than once. - If true indicates that no edge is stored twice. Users are encouraged to - consider and use the half_edge_data_structure instead as this will work - towards achieving a cleaner graph-based description if relevant and possible. + If true, indicates that no edge is stored more than once. + + Users are encouraged to consider using a half_edge_data_structure instead. + + + + + If true, indicates that no face is stored more than once. - Specifies for each face which winding order was used if any: diff --git a/base_classes/NXcg_geodesic_mesh.nxdl.xml b/base_classes/NXcg_geodesic_mesh.nxdl.xml index 6314db3427..244290a81b 100644 --- a/base_classes/NXcg_geodesic_mesh.nxdl.xml +++ b/base_classes/NXcg_geodesic_mesh.nxdl.xml @@ -2,9 +2,9 @@ - - - Container to hold different coordinate systems conventions. - - It is the purpose of this base class to define these conventions and - offer a place to store mappings between different coordinate systems - which are relevant for the interpretation of the data described - by the application definition and base class instances. - - For each Cartesian coordinate system users should use a set of - NXtransformations: - - * These should define the three base vectors. - * The location of the origin. - * The affine transformations which bring each axis of this coordinate system - into registration with the McStas coordinate system. - * Equally, affine transformations should be given for the inverse mapping. - - As an example one may take an experiment or computer simulation where - there is a laboratory (lab) coordinate system, a sample/specimen coordinate - system, a crystal coordinate system, and additional coordinate systems, - which are eventually attached to components of the instrument. - - If no additional transformation is specified in this group or if an - instance of an NXcoordinate_system_set is absent it should be assumed - the so-called McStas coordinate system is used. - - Many application definitions in NeXus refer to this `McStas <https://mailman2.mcstas.org/pipermail/mcstas-users/2021q2/001431.html>`_ coordinate system. - This is a Cartesian coordinate system whose z axis points along the neutron - propagation axis. The systems y axis is vertical up, while the x axis points - left when looking along the z-axis. Thus, McStas is a right-handed coordinate system. - - Within each NXtransformations a depends_on section is required. The depends_on - field specifies if the coordinate system is the root/reference - (which is indicated by writing "." in the depends_on section.) - - - - - A group of transformations which specify: - - * Three base vectors of the coordinate system. - * Origin of the coordinate system. - * A depends_on keyword. Its value can be "." or the name of an - NXtransformations instance which needs to exist in the - NXcoordinate_system_set instance. - * If the coordinate system is the reference one it has to be named - reference. - - In case of having more than one NXtransformations there has to be for - each additional coordinate system, i.e. the one not the reference: - - * A set of translations and rotations which map each base vector to the reference. - * A set of translations and rotations which map each reference base vector - to the coordinate system. - - The NXtransformations for these mappings need to be formatted - according to the descriptions in NXtransformations. - - - - - - - diff --git a/base_classes/NXcorrector_cs.nxdl.xml b/base_classes/NXcorrector_cs.nxdl.xml deleted file mode 100644 index fcca05d7a1..0000000000 --- a/base_classes/NXcorrector_cs.nxdl.xml +++ /dev/null @@ -1,95 +0,0 @@ - - - - - - Corrector for aberrations in an electron microscope. - - Different technology partners use different naming schemes and models - for quantifying the aberration coefficients. - - The corrector in an electron microscope is composed of multiple lenses and - multipole stigmators with vendor-specific details which are often undisclosed. - - - - Was the corrector used? - - - - - Given name/alias. - - - - - - Ideally, a (globally) unique persistent identifier, link, - or text to a resource which gives further details. If this does not exist - a free-text field to report further details about the corrector. - - - - - - Specific information about the concrete alignment procedure which is a - process during which the corrector is configured to enable a calibrated - usage of the microscope. - - - - Discouraged free-text field to add further details about the alignment - procedure. - - - - - The outer tilt angle of the beam in tableau aquisition. - - - - - The exposure time of the single tilt images. - - - - - The factor of enlargement of the apparent size, - not physical size, of an object. - - - - - Place for storing measured or estimated aberrations (for each image or final). - - - - - - - - - - diff --git a/base_classes/NXcs_computer.nxdl.xml b/base_classes/NXcs_computer.nxdl.xml deleted file mode 100644 index e6f1275ff5..0000000000 --- a/base_classes/NXcs_computer.nxdl.xml +++ /dev/null @@ -1,80 +0,0 @@ - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Computer science description of a set of computing nodes. - - - - Given name/alias to the computing system, e.g. MyDesktop. - - - - - Name of the operating system, e.g. Windows, Linux, Mac, Android. - - - - Version plus build number, commit hash, or description of an ever - persistent resource where the source code of the program and build - instructions can be found so that the program can be configured in - such a manner that the result file is ideally recreatable yielding - the same results. - - - - - - - Ideally a (globally) unique persistent identifier of the computer, i.e. - the Universally Unique Identifier (UUID) of the computing node. - - - - - - A list of physical processing units (can be multi-core chips). - - - - - A list of physical coprocessor/graphic cards/accelerator units. - - - - - Details about the memory sub-system. - - - - - Details about the I/O sub-system. - - - diff --git a/base_classes/NXcs_filter_boolean_mask.nxdl.xml b/base_classes/NXcs_filter_boolean_mask.nxdl.xml deleted file mode 100644 index 2ee961538f..0000000000 --- a/base_classes/NXcs_filter_boolean_mask.nxdl.xml +++ /dev/null @@ -1,104 +0,0 @@ - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Number of entries (e.g. number of points or objects). - - - - - Number of bits assumed for the container datatype used. - - - - - Length of mask considering the eventual need for padding. - - - - - Computer science base class for packing and unpacking booleans. - - One use case is processing of object sets (like point cloud data). - When one applies e.g. a spatial filter to a set of points to define which - points are analyzed and which not, it is useful to document which points were - taken. One can store this information in a compact manner with an array of - boolean values. If the value is True the point is taken, else it is not. - - If the points are identified by an array of integer identifiers and an - arbitrary spatial filtering, the boolean array will be filled with True and False - values in an arbitrary manner. Especially when the number of points is large, - for instance several thousands and more, some situations can be more efficiently - stored if one would not store the boolean array but just list the identifiers - of the points taken. For instance if within a set of 1000 points only one point is - taken, it would take (naively) 4000 bits to store the array but only 32 bits - to store e.g. the ID of that taken point. Of course the 4000 bit field is so - sparse that it could be compressed resulting also in a substantial reduction - of the storage demands. Therefore boolean masks are useful compact descriptions - to store information about set memberships in a compact manner. - In general it is true, though, that which representation is best, i.e. - most compact (especially when compressed) depends strongly on occupation of - the array. - - This base class just bookkeeps metadata to inform software about necessary - modulo operations to decode the set membership of each object. This is useful - because the number of objects not necessarily is an integer multiple of the bit depth. - - - - Number of objects represented by the mask. - - - - - Number of bits assumed matching on a default datatype. - (e.g. 8 bits for a C-style uint8). - - - - - The unsigned integer array representing the content of the mask. - If padding is used the padded bits have to be set to 0. - - - - - - - - Link to/or array of identifiers for the objects. The decoded mask is - interpreted consecutively, i.e. the first bit in the mask matches - to the first identifier, the second bit in the mask to the second - identifier and so on and so forth. - - - - - - diff --git a/base_classes/NXcs_prng.nxdl.xml b/base_classes/NXcs_prng.nxdl.xml deleted file mode 100644 index 83d4e003a7..0000000000 --- a/base_classes/NXcs_prng.nxdl.xml +++ /dev/null @@ -1,85 +0,0 @@ - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Computer science description of pseudo-random number generator. - - The purpose of such metadata is to identify if exactly the same sequence - can be reproduced, like for a PRNG or not (for a true physically random source). - - - - Different approaches for generating random numbers with a computer exists. - Some use a dedicated physical device where the state is unpredictable (physically). - Some use a mangling of the system clock (system_clock), where also without - additional pieces of information the sequence is not reproducible. - Some use so-called pseudo-random number generator (PRNG) are used. - These are algorithms which yield a deterministic sequence of practically - randomly appearing numbers. These algorithms different in their quality in - how close the resulting sequences are random. - Nowadays one of the most commonly used algorithm is - the MersenneTwister (mt19937). - - - - - - - - - - - Name of the PRNG implementation and version. If such information is not - available or if the PRNG type was set to other the DOI to the publication - or the source code should be given. - - - - Version and build number, or commit hash. - - - - - - Parameter of the PRNG controlling its initialization and thus the specific - sequence of numbers it generates. - - - - - - Number of initial draws from the PRNG which are discarded in an effort - to equilibrate the sequence and make it thus to statistically more random. - If no warmup was performed or if warmup procedures are unclear, - users should set the value to zero. - - - - diff --git a/base_classes/NXcs_profiling.nxdl.xml b/base_classes/NXcs_profiling.nxdl.xml deleted file mode 100644 index d289ef2923..0000000000 --- a/base_classes/NXcs_profiling.nxdl.xml +++ /dev/null @@ -1,149 +0,0 @@ - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Computer science description for summary performance/profiling data of an application. - - Performance monitoring and benchmarking of software is a task where questions - can be asked at various levels of detail. In general, there are three main - contributions to performance: - - * Hardware capabilities and configuration - * Software configuration and capabilities - * Dynamic effects of the system in operation and the system working together - with eventually multiple computers, especially when these have to exchange - information across a network. - - At the most basic level users may wish to document how long e.g. a data - analysis with a scientific software (app). - A frequent idea is here to judge how critical the effect is on the workflow - of the scientists, i.e. is the analysis possible in a few seconds or would it - take days if I were to run this analysis on a comparable machine. In this case, - mainly the order of magnitude is relevant, as well as how this can be achieved - with using parallelization (i.e. reporting the number of CPU and GPU resources - used, the number of processes and/or threads, and basic details about the - computing node/computer. - - At more advanced levels benchmarks may go as deep as detailed temporal tracking - of individual processor instructions, their relation to other instructions, the - state of call stacks, in short eventually the entire app execution history - and hardware state history. Such analyses are mainly used for performance - optimization as well as for tracking bugs and other development purposes. - Specialized software exists which documents such performance data in - specifically-formatted event log files or databases. - - This base class cannot and should not replace these specific solutions. - Instead, the intention of the base class is to serve scientists at the - basic level to enable simple monitoring of performance data and log profiling - data of key algorithmic steps or parts of computational workflows, so that - these pieces of information can guide users which order of magnitude differences - should be expected or not. - - Developers of application definitions should add additional fields and - references to e.g. more detailed performance data to which they wish to link - the metadata in this base class. - - - - - Path to the directory from which the tool was called. - - - - - Command line call with arguments if applicable. - - - - - ISO 8601 time code with local time zone offset to UTC information - included when the app was started. - - - - - ISO 8601 time code with local time zone offset to UTC information - included when the app terminated or crashed. - - - - - Wall-clock time how long the app execution took. This may be in principle - end_time minus start_time; however usage of eventually more precise timers - may warrant to use a finer temporal discretization, - and thus demand a more precise record of the wall-clock time. - - - - - Qualifier which specifies with how many nominal processes the app was - invoked. The main idea behind this field, for instance for app using a - Message Passing Interface parallelization is to communicate how many - processes were used. - - For sequentially running apps number_of_processes and number_of_threads - is 1. If the app uses exclusively GPU parallelization number_of_gpus - can be larger than 1. If no GPU is used number_of_gpus is 0 even though - the hardware may have GPUs installed, thus indicating these were not - used though. - - - - - Qualifier with how many nominal threads were accessible to the app at - runtime. Specifically here the maximum number of threads used for the - high-level threading library used (e.g. OMP_NUM_THREADS), posix. - - - - - Qualifier with how many nominal GPUs the app was invoked at runtime. - - - - - - A collection with one or more computing nodes each with own resources. - This can be as simple as a laptop or the nodes of a cluster computer. - - - - - A collection of individual profiling event data which detail e.g. how - much time the app took for certain computational steps and/or how much - memory was consumed during these operations. - - - - diff --git a/base_classes/NXcs_profiling_event.nxdl.xml b/base_classes/NXcs_profiling_event.nxdl.xml deleted file mode 100644 index 3b2d4be1db..0000000000 --- a/base_classes/NXcs_profiling_event.nxdl.xml +++ /dev/null @@ -1,95 +0,0 @@ - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Number of processes. - - - - - Computer science description of a profiling event. - - - - ISO 8601 time code with local time zone offset to UTC information - included when the event tracking started. - - - - - ISO 8601 time code with local time zone offset to UTC information - included when the event tracking ended. - - - - - Free-text description what was monitored/executed during the event. - - - - - Wall-clock time how long the event took. This may be in principle - end_time minus start_time; however usage of eventually more precise timers - may warrant to use a finer temporal discretization, - and thus demand a more precise record of the wall-clock time. - Elapsed time may contain time portions where resources were idling. - - - - - Number of processes used (max) during the execution of this event. - - - - - Number of threads used (max) during the execution of this event. - - - - - Number of GPUs used (max) during the execution of this event. - - - - - Maximum amount of virtual memory allocated per process during the event. - - - - - - - - Maximum amount of resident memory allocated per process during the event. - - - - - - diff --git a/base_classes/NXdeflector.nxdl.xml b/base_classes/NXdeflector.nxdl.xml deleted file mode 100644 index 01db33f5cf..0000000000 --- a/base_classes/NXdeflector.nxdl.xml +++ /dev/null @@ -1,92 +0,0 @@ - - - - - - Deflectors as they are used e.g. in an electron analyser. - - - - Qualitative type of deflector with respect to the number of pole pieces - - - - - - - - - - - Colloquial or short name for the deflector. For manufacturer names and - identifiers use respective manufacturer fields. - - - - - Name of the manufacturer who built/constructed the deflector. - - - - - Hardware name, hash identifier, or serial number that was given by the - manufacturer to identify the deflector. - - - - - Ideally an identifier, persistent link, or free text which gives further details - about the deflector. - - - - - Excitation voltage of the deflector. For dipoles it is a single number. For - higher orders, it is an array. - - - - - Excitation current of the deflector. For dipoles it is a single number. For - higher orders, it is an array. - - - - - Specifies the position of the deflector by pointing to the last transformation - in the transformation chain in the NXtransformations group. - - - - - Collection of axis-based translations and rotations to describe the location and - geometry of the deflector as a component in the instrument. Conventions from the - NXtransformations base class are used. In principle, the McStas coordinate - system is used. The first transformation has to point either to another - component of the system or . (for pointing to the reference frame) to relate it - relative to the experimental setup. Typically, the components of a system should - all be related relative to each other and only one component should relate to - the reference coordinate system. - - - diff --git a/base_classes/NXebeam_column.nxdl.xml b/base_classes/NXebeam_column.nxdl.xml deleted file mode 100644 index e31f22fbcc..0000000000 --- a/base_classes/NXebeam_column.nxdl.xml +++ /dev/null @@ -1,109 +0,0 @@ - - - - - - - Container for components to form a controlled beam in electron microscopy. - - - - - The source which creates the electron beam. - - - - Given name/alias. - - - - - - Voltage relevant to compute the energy of the electrons - immediately after they left the gun. - - - - - Type of radiation. - - - - - - - - Emitter type used to create the beam. - - If the emitter type is other, give further details - in the description field. - - - - - - - - - - - Material of which the emitter is build, e.g. the filament material. - - - - - - Ideally, a (globally) unique persistent identifier, link, - or text to a resource which gives further details. - - - - - - Affine transformation which detail the arrangement in the - microscope relative to the optical axis and beam path. - - - - - - - - - - - A sensor used to monitor an external or internal condition. - - - - - Individual ocharacterization results for the position, shape, - and characteristics of the electron beam. - - NXtransformations should be used to specify the location - of the position at which the beam was probed. - - - diff --git a/base_classes/NXem_ebsd.nxdl.xml b/base_classes/NXem_ebsd.nxdl.xml deleted file mode 100644 index aa3dd46be3..0000000000 --- a/base_classes/NXem_ebsd.nxdl.xml +++ /dev/null @@ -1,1926 +0,0 @@ - - - - - - - - - - Number of arguments per orientation for given parameterization. - - - - - Number of scan points. - - - - - Number of pixel along the slowest changing dimension for a rediscretized, i.e. - standardized default orientation mapping. - - - - - Number of pixel along slow changing dimension for a rediscretized i.e. - standardized default orientation mapping. - - - - - Number of pixel along fast changing dimension for a rediscretized i.e. - standardized default orientation mapping. - - - - - - Application definition for collecting and indexing Kikuchi pattern into orientation maps. - - This NXem_ebsd application is a proposal how to represent data, metadata, and - connections between these for the research field of electron microscopy. - More specifically, exemplified here for electron backscatter diffraction (EBSD). - The application definition solves two key documentation issues which are missing - so far to document provenance of data and metadata in the field of EBSD. - The application definition can be an example that is relevant for related - workflows in orientation microscopy. - - Firstly, an instance of NXem_ebsd (such as a NeXus/HDF5 file which is formatted - according to the NXem_ebsd application definition) stores the connection between - the microscope session and the key datasets which are considered typically results - of the various processing steps involved when working with EBSD data. - - Different groups in this application definition make connections to data artifacts - which were collected when working with electron microscopes via the NXem partner - application definition. Using a file which stores information according to the - NXem application definition has the benefit that it connects the sample, references - to the sample processing, the user operating the microscope, details about the - microscope session, and details about the acquistion and eventual indexing of - Kikuchi pattern, associated overview images, like secondary electron or - backscattered electron images of the region-of-interest probed and many - more pieces of information. - - Secondly, this NXem_ebsd application definition connects and stores the conventions - and reference frames which were used and are the key to mathematically correctly - interpret every EBSD result. Otherwise, results would be ripped out of their - context, as it is the situation with many traditional studies where EBSD data were - indexed on-the-fly and shared with the community only via sharing the results file - with some technology-partner-specific file but leaving important conventions out - or relying on the assumptions that colleagues know these even though multiple - definitions are possible. - - This application definition covers experiments with one-, two-dimensional, and - so-called three-dimensional EBSD datasets. The third dimension is either time - (in the case of quasi in-situ experiments) or space (in the case of serial- - sectioning) methods where a combination of mechanical or ion milling is used - repetitively to measure the same region-of-interest at different depth increments. - Material removal can be achieved with electron or ion polishing, using manual - steps or using automated equipment like a robot system. - - Three-dimensional experiments require to follow a sequence of specimen, surface - preparation, and data collection steps. By nature these methods are destructive - in that they either require the removal of the previously measured material region - or that the sample surface can degrade due to e.g. contamination or other - electron-matter interaction. - - For three-dimensional EBSD, multiple two-dimensional EBSD orientation mappings are - combined into one reconstructed stack. That is serial-sectioning is mainly a - computational workflow. Users collect data for each serial sectioning step - via an experiment. This assures that data for associated microscope sessions - and steps of data processing stay connected and contextualized. - - Eventual tomography methods also use such a workflow because first diffraction - images are collected (e.g. with X-ray) and then these imagres are indexed and - computed into a 3D orientation mapping. The here proposed NXem_ebsd application - definition contains conceptual ideas how this splitting between measurement and - post-processing can be granularized also for such X-ray-based techniques, whether - it be 3DXRD or HEDM. - - - - - An at least as strong as SHA256 hashvalue of the file - that specifies the application definition. - - - - - - NeXus NXDL schema to which this file conforms. - - - - - - - - Ideally, a (globally) unique persistent identifier - for referring to this workflow. - - The identifier is usually defined/issued by the facility, laboratory, - or the principle investigator. The identifier enables to link - workflows/experiments to e.g. proposals. - - - - - Free-text description about the workflow. - - Users are strongly advised to detail the sample history in the respective - field and fill rather as completely as possible the fields of the application - definition behind instead of filling in these details into the experiment_description - free-text description field. - - - - - ISO 8601 time code with local time zone offset to UTC information - included when the processing of the workflow started. - If the application demands that time codes in this section of the - application definition should only be used for specifying when the - workflow was executed - and the exact duration is not relevant - - this start_time field should be used. - - Often though it is useful to specify a time interval with specifying - both start_time and end_time to allow for more detailed bookkeeping - and interpretation of the workflow. - - - - - ISO 8601 time code with local time zone offset to UTC included - when the processing of the workflow ended. - - - - - Program which was used for creating the file instance which is - formatted according to the NXem_ebsd application definition. - - - - - - - - Contact information and eventually details of at least one person - involved in performing the workflow. This can be the principle investigator - who performed this experiment. Adding multiple users if relevant is - recommended. - - - - Given (first) name and surname of the user. - - - - - Name of the affiliation of the user at the point in time - when the experiment was performed. - - - - - Postal address of the affiliation. - - - - - Email address of the user at the point in time when the experiment - was performed. Writing the most permanently used email is recommended. - - - - - Globally unique identifier of the user as offered by services - like ORCID or ResearcherID. If this field is field the specific - service should also be written in orcid_platform - - - - - Name of the OrcID or ResearcherID where the account - under orcid is registered. - - - - - (Business) (tele)phone number of the user at the point - in time when the experiment was performed. - - - - - Which role does the user have in the place and at the point - in time when the experiment was performed? Technician operating - the microscope. Student, postdoc, principle investigator, guest - are common examples. - - - - - Account name that is associated with the user - in social media platforms. - - - - - Name of the social media platform where the account - under social_media_name is registered. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Details about simulations for Kikuchi pattern using kinematic or dynamic - diffraction theory. Usually, the output of such computer simulations are - spherical Kikuchi images which only when projected or observed in some - region-of-interest will represent a set of rectangular Kikuchi pattern - with the same rectangular shape and image size. - - Therefore, these pattern should be stored. The spherical diffraction - pattern can be stored as a set of triangulated geodesic meshes. - The rectangular patterns should be stored as NXimage_set_em_kikuchi stack. - - Do not store pattern in the simulation group if they - have been measured are not simulated. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The experiment group captures relevant details about the conditions of - and the tools used for collecting the Kikuchi diffraction pattern. - - The most frequently collected EBSD data are captured as rectangular ROIs - composed from square or hexagonally-shaped pixels. Substantially less - frequently, because such experiments are more costly and technically - demanding, correlated experiments are performed. - - One important class of such correlated experiments are the so-called - (quasi) in-situ experiments. Here the same or nearly the same ROI is - analyzed via a cycles of thermomechanical treatment, sample preparation, - measurement, on-the-fly-indexing. Phenomena investigated like this are - recrystallization, strain accumulation, material damage. - Post-processing is required to correlate and reidentify eventual - features or local ROIs across several orientation maps. - - Another important class of correlated experiments are the so-called - serial-sectioning experiments. Here the same sample is repetitively measured - and polished to create a stack of orientation data which can be reconstructed - to a three-dimensional volume ROI. - - - - Physical time since the beginning of a timestamp that is required to be - same for all experiments in the set. The purpose of this marker is - to identify how all experiments in the set have have to be arranged - sequentially based on the time elapsed. - The time is relevant to sort e.g. experiments of consecutive quasi - in-situ experiments where a measurement was e.g. taken after 0 minutes - of annealing, 30 minutes, 6 hours, or 24 hours of annealing. - - - - - Transformation which details where the region-of-interest described under - indexing is located in absolute coordinates and rotation with respect - to which coordinate system. - - - - - - The EBSD system, including components like the electron gun, pole-piece, - stage tilting, EBSD detector, and the gnomonic projection have to be - calibrated to achieve reliable results. Specifically, the gnomonic projection - has to be calibrated. - - In most practical cases, especially in engineering, there is a substantially - larger number of sessions where such a calibrated system is used assuming - that somebody has properly calibrated the system rather than that the user - actively recalibrates it or is even allowed to do so. - Especially the projection geometry has to calibrated which is usually - achieved with measuring silicon, quartz or standards, and comparing - against simulated diffraction pattern. - - In the first case, the user assumes that the principle geometry of the - hardware components and the settings in the control and EBSD pattern - acquisition software are calibrated. Consequently, users pick from an - existent library of phase candidates. One example are the CRY or CIF - files of the classical HKL/Channel 5/Flamenco software products. - Each entry of the library of such phase candidates in this NeXus proposal - is represented by one NXem_ebsd_crystal_structure_model base class. - For each phase an instance of this base class is to be used to store - crystallographic and simulation-relevant data. - - Indexing is a data processing step performed after/during the beam scans - the specimen (depends on configuration). Users load the specimen, and first - collect a coarse image of the surface. Next, an approximate value for the - calibrated working distance is chosen and the stage tilted. - Users then may configure the microscope for collecting higher quality data - and push in the EBSD detector. Subsequently, they fine tune the illumination - and aberration settings and select one or multiple ROIs to machine off. - The on-the-fly indexing parameter are defined and usually the automated - measurement queue started. - - Nowadays, this is usually an automated/unsupervised process. The pattern - collection runs during the allocated session time slot which the user has - booked ends or when the queue finishes prematurely. Kikuchi pattern surplus - eventually multi-modal detector signals are collected and usually indexed - on-the-fly. The Kikuchi patterns may or not be deleted directly after a - solution was found (on-the-fly) so Kikuchi pattern are not always stored. - - Results files are in many labs afterwards copied automatically - for archival purposes to certain storage locations. The result of such an - EBSD measurement/experiment is a set of usually proprietary or open files - from technology partners (microscope and EBSD detector manufacturers). - - In the second case, the system is being calibrated during the session - using standards (silicon, quartz, or other common specimens). - There is usually one person in each lab responsible for doing such - calibrations. Important is that often this person or technician(s) are also - in charge of configuring the graphical user interface and software - with which most users control and perform their analyses. - For EBSD this has key implications because, taking TSL OIM/EDAX as an example, - the conventions how orientations are stored is affected by how reference frames - are set up and this setup is made at the level of the GUI software. - Unfortunately, these pieces of information are not necessarily stored - in the results files. In effect, key conventions become disconnected - from the data so it remains the users personal obligation to remember these - settings, write them down in the lab notebook, or these metadata get lost. - All these issues are a motivation and problem which NXem_ebsd solves. - - - - - A link/cross reference to an existent instance of NXem_ebsd with ideally - an associated instance of NXem detailed under measurement which informs - about the calibration procedures. - - - - Commit identifying this resource. - - - - - - Path which resolves which specific NXimage_set_em_kikuchi instance - was used as the raw data to the EBSD data (post)-processing workflow - when performing the calibration. - - - - - - - Relevant result of the session at the microscope for this experiment - which enables to connect the measurement of the Kikuchi pattern and - their processing into orientation microscopy maps. - - - - - Name or link to an existent instance of an EBSD raw dataset ideally - as an instance of an NXem application definition which has at least - one NXimage_set_em_kikuchi instance i.e. one stack of Kikuchi pattern. - The path to this instance in the origin has to be specified under path. - - When NXem is not used or the aim is to rather explore first how - community-specific files with EBSD data, such as ANG, CPR, or HDF5- - based formats can be parsed from, inject here the name of that file. - - The em_om parser will currently not interpret the majority of the - many system- and technique-specific metadata which come with the - files from e.g. technology partners. This is because the current - culture in the EBSD community is that many of the metadata fields - are neither in all cases fully documented nor use a standardized - vocabulary although many people understand terms from different - implementations and how these metadata can likely be compared to - one another. - - In addition, it is common practice in the research field of EBSD that - users transcode their raw data into other (often text-based or HDF5) - files with custom formatting to realize an information transfer - between specific software tools including commercial software from - technology partner, custom scripts in Matlab using tools like MTex, - or Python scripting with tools like hyperspy, pyxem, orix, diffsims, - kikuchipy, or EBSD data stack alignment tools like DREAM.3D. - We have opted that in the first iteration this implementation of a - RDMS-agnostic FAIR data schema for EBSD that we discard these metadata - because these ad hoc file formats are not designed to communicate - also specifically and most importantly the eventually different context - of the metadata. - Another reason for this choice was also to emphasize that in fact such - challenges do exist in the community and thus pointing them out may - support the discussion to arrive at eventually more complete solutions. - As developing these solutions should not be our authority and necessarily - demands feedback from the technology partners, we have opted for this - intermediate approach to stimulate discussion. - - - - Commit or e.g. at least SHA256 checksum identifying this resource. - - - - - - Path which resolves which specific NXimage_set_em_kikuchi instance - was used as the raw data to this EBSD data (post)-processing workflow. - - - - - - - - OIM, orientation imaging microscopy. Post-processing of the Kikuchi - patterns to obtain orientation per phase model and scan point. - Fundamentally different algorithms can be used to index EBSD/EBSP pattern. - - Common is that pattern indexing is a computational step of comparing - simulated with measured diffraction pattern. Quality descriptors are defined - based on which an indexing algorithm yields a quantitative measure of - how similar measured and assumed/simulated pattern are, and thus if - no, one, or multiple so-called solutions were found. - - Assumed or simulated pattern use kinematical or dynamical electron - diffraction theory. Hough transform (which is essentially a discretized - Radon transform, for details see e.g A short introduction to the Radon - and Hough transforms and how they relate by M. van Ginkel et al.). - Recently, dictionary-based indexing methods are increasingly becoming used - partly driven by the move to use artificial intelligence algorithms. - - An inspection of publicly available EBSD datasets with an open-source - license which are available on Zenodo was performed prior to implementing - of the associated em_om parser for NXem_ebsd. This analysis revealed that - EBSD data are in most cases stored in two ways: Case one was via a file in - formats from technology partners. Examples are binary formats like OSC, - H5OINA, OIP, EBSP, and many derived text-based formats like CPR, CRC, ANG, - CTF, HKL and more. Recently, there is trend towards using HDF5-based formats. - - These files contain some result and metadata to the numerical steps and the - computational workflow which was performed to index Kikuchi pattern - on-the-fly. Examples of metadata include scan point positions, indexing - solutions per scan point, some quality descriptors for the solutions, - as well as crystal structure and phase metadata. - - Case two were raw pattern in some custom format, often text-based with - some but in general no conclusive and interoperable representation of all - relevant metadata. - Often it remains unclear what individual fields and data arrays of these - fields resolve and/or mean conceptually. For some fields, publications were - referred to. However, software tools change over time and thus which specific - data end in a file and which specific conceptual information is behind - these data can change with software versions. - - Other cases were storing results of custom post-processing steps and - associated Kikuchi pattern. Testing of advanced indexing, pseudo-symmetry - resolving methods, i.e. any sort of prototyping or alternative indexing - strategies so far seem to require some flexibility for implementing - rapid prototypic capabilities. The drawback of this is that such results - come formatted on a case-by-case basis and are thus not interoperable. - - Therefore, we first need to collect how these files have been generated - and which metadata in these files (or database entries) represent - which pieces of information conceptually. Ideally, one would do so by - creating a complete set of information in e.g. an NXem application definition, - such as a log of timestamped events and processing steps, metadata and data. - Eventually even interactions with the graphical user interface of commercial - software during the microscope session should be stored and become a - part of the application definition. - - Such a set of pieces of information could then be used via reading directly - for the NXem application definition. However, in most cases such a data - representation is not available yet. - - - - - Therefore, the on_the_fly_indexing group stores which source_file contains - the results of the on-the-fly indexing. For commercial systems these files - can be e.g. ANG, CPR/CRC, H5OINA, OSC. It is possible that the file or - database entry which is referred to under origin is the same as the one - under a given acquisition/origin in one of the experiment groups. - This is because some commercial file formats make no clear distinction - between which metadata are acquisition and/or indexing metadata. - - - - Commercial program which was used to index the EBSD data - incrementally after they have been captured and while the - microscope was capturing (on-the-fly). This is the usual - production workflow how EBSD data are collected in - materials engineering, in industry, and academia. - - - - - - - - Name of the file from which data relevant for creating default plots - were taken in the case that the data in the experiment group were - indexed on-the-fly. - - - - Hash of that file. - - - - - - TBD, path which resolves which specific NXimage_set_em_kikuchi instance - was used as the raw data to this EBSD data (post)-processing workflow - when performing the calibration. - - - - - - Principal algorithm used for indexing. - - - - - - - - - - - - Details about the background correction applied to each Kikuchi pattern. - - - - - - - Binning i.e. downsampling of the pattern. - - - - - - - Specific parameter relevant only for certain algorithms used - - - - - - - - - - - - - - - - - - - - - - - - - - - Which return value did the indexing algorithm yield for each scan point. - Practically useful is to use an uint8 mask. - - * 0 - Not analyzed - * 1 - Too high angular deviation - * 2 - No solution - * 100 - Success - * 255 - Unexpected errors - - - - - - - - - How many phases i.e. crystal structure models were used to index each - scan point if any? Let's assume an example to explain how this field - should be used: In the simplest case users collected one pattern for - each scan point and have indexed using one phase, i.e. one instance - of an NXem_ebsd_crystal_structure_model. - - In another example users may have skipped some scan points (not indexed) - them at all) and/or used differing numbers of phases for different scan - points. - - The cumulated of this array decodes how phase_identifier and phase_matching - arrays have to be interpreted. In the simplest case (one pattern per scan - point, and all scan points indexed using that same single phase model), - phase_identifier has as many entries as scan points - and phase_matching has also as many entries as scan points. - - - - - - - - The array n_phases_per_scan_point details how the phase_identifier - and the phase_matching arrays have to be interpreted. - - For the example with a single phase phase_identifier has trivial - values either 0 (no solution) or 1 (solution matching - sufficiently significant with the model for phase 1). - - When there are multiple phases, it is possible (although not frequently - needed) that a pattern matches eventually (not equally well) sufficiently - significant with multiple pattern. This can especially happen in cases of - pseudosymmetry and more frequently with an improperly calibrated system - or false or inaccurate phase models e.g. (ferrite, austenite). - Having such field is especially relevant for recent machine learning - or dictionary based indexing schemes because in combination with - phase_matching these fields communicate the results in a model-agnostic - way. - - Depending on the n_phases_per_scan_point value phase_identifier and - phase_matching arrays represent a collection of concatenated tuples, - which are organized in sequence: The solutions for the 0-th scan point, - the 1-th scan point, the n_sc - 1 th scan point and omitting tuples - for those scan points with no phases according to n_phases_per_scan_point - - - - - - - - One-dimensional array, pattern by pattern labelling the solutions found. - The array n_phases_per_scan_point has to be specified because it details - how the phase_identifier and the phase_matching arrays have to be interpreted. - See documentation of phase_identifier for further details. - - - - - - - - Phase_matching is a descriptor for how well the solution matches or not. - Examples can be confidence index (ci), mean angular deviation (mad), - some AI-based matching probability (other), i.e. the details are implementation-specific. - - - - - - - - - - - How are orientations parameterized? Inspect euler_angle_convention - in case of using euler to clarify the sequence of rotations assumed. - - - - - - - - - - - - Matrix of parameterized orientations identified. The slow dimension - iterates of the individual solutions as defined by n_phases_per_scan_point. - Values for phases without a solution should be correctly identified as - IEEE NaN. - - - - - - - - em_lab/ebeam_deflector to retrieve the actual scan positions -although this would be much cleaner--> - - Matrix of calibrated centre positions of each scan point - in the sample surface reference system. - - - - - - - - - - - Fraction of successfully indexed pattern - of the set averaged over entire set. - - - - - - An overview of the entire area which was scanned. - For details about what defines the image contrast - inspect descriptor. - - - - Descriptor representing the image contrast. - - - - - - - - - - Container holding a default plot of the region on the sample - investigated with EBSD. - - - - - - - - - - Descriptor values displaying the ROI. - - - - - - - - - Signal - - - - - - Calibrated center of mass of the pixel along the slow axis. - - - - - - - Label for the y axis - - - - - - Calibrated center of mass of the pixel along the fast axis. - - - - - - - Label for the x axis - - - - - - - - Default inverse pole figure (IPF) plot of the data specific for each - phase. No ipf_mapID instances for non-indexed scan points as these are - by definition assigned the null phase with phase_identifier 0. - - The IPF mapping is interpolated from the scan point data mapping - onto a rectangular domain with square pixels and the - orientations colored according to the coloring scheme used in the - respective ipf_color_modelID/program. - - The main purpose of the ipf_mapID group is not to keep raw data or - scan point related data but offer a default way how a research data - management system can display a preview of the dataset so that users - working with the RDMS can get an overview of the dataset. - - This matches the first aim of NXem_ebsd which is foremost to bring - colleagues and users of EBSD together to discuss which pieces of information - need to be stored together. We are convinced a step-by-step design and - community-driven discussion about which pieces of information should - and/or need to be included is a practical strategy to work towards an - interoperable description and data model for exchanging - data from EBSD between different tools and research data management - systems (RDMS). - - With this design the individual RDMS solutions and tools can still continue - to support specific custom data analyses workflow and routes but at least - there is then one common notation of understanding whereby also users - not necessarily expert in all the details of the EBSD story can understand - better these data and thus eventually this can motivate data reuse and - repurposing. - - It is important to mention that we cannot assume, at least for now, - that the parser which writes to an NXem_ebsd-compliant file is also - responsible or capable at all of computing the inverse pole figure - color keys and maps itself. This cannot be assumed working because - this mapping of orientation data uses involved mathematical algorithms - and functions which not every tools used in the EBSD community is capable - of using or is for sure not using in exactly the same way. - - Currently, we assume it is the responsibilty of the tool used which - generated the data under on_the_fly_indexing to compute these - plots and deliver these to the parser. - - Specific case studies have been explored by the experiment team of - Area B of the FAIRmat project to realize and implement such mapping. - - The first case study uses the H5OINA format and the pyxem/orix library. - As orix is a Python library, the coloring is performed by the em_om parser. - - The second case study uses MTex and its EBSD color coding model. - As MTex is a Matlab tool, an intermediate format is written from MTex - first which stores these pieces of information. The parser then pulls - these data from the intermediate Matlab-agnostic representation and - supplements the file with missing pieces of information as it is - required by NXem_ebsd. - - The third case study shows how a generic set of Kikuchi pattern - can be loaded with the em_om parser. The pattern are loaded directly - from a ZIP file and mapped to an simulation image section for now. - - The fourth case study uses the DREAM.3D package which provides an own - set of EBSD data post-processing procedures. DREAM.3D documents the - processing steps with a pipeline file which is stored inside DREAM.3D - output files. In this case study, the parser reads the DREAM.3D file - and maps data relevant from the perspective of NXem_ebsd plus adds - relevant IPF color maps as they were computed by DREAM.3D. - Given that in this case the origin of the data is the DREAM.3D file - again provenance is kept and more details can be followed upon when - resolving origin. - - These examples offer a first set of suggestions on how to make EBSD - data injectable into research data management system using schemes - which themselves are agnostic to the specific RDMS and interoperable. - Steps of collecting the raw data and post-processing these with custom - scripts like MTex or commercial tools so far are mainly undocumented. - The limitation is that a program which consumes results or dump files - from these tools may not have necessarily all the sufficient information - available to check if the injected orientation data and color models - are matching the conventions which a user or automated system has - injected into an electronic lab notebook from which currently the em_om - parser collects the conventions and stores them into this NXem_ebsd instance. - The immediate benefit of the here presented NXem_ebsd concept though - is that the conventions and reference frame definitions are expected - in an ELN-agnostic representation to make NXem_ebsd a generally useful - data scheme for EBSD. - - Ideally, the em_om parser would load convention-compliant EBSD data - and use subsequently a community library to transcode/convert orientation - conventions and parameterized orientation values. Thereafter, convention- - compliant default plot(s) could be created that would be truely interoperable. - - However, given the variety of post-processing tools available surplus - the fact that these are not usually executed along standardized - post-processing workflows which perform exactly the same algorithmic steps, - this is currently not a practically implementable option. Indeed, first - developers who wish to implement this would first have to create a library - for performing such tasks, mapping generally between conventions, - i.e. map and rotate coordinate systems at the parser level. - - The unfortunate situation in EBSD is that due to historical reasons - and competitive strategies, different players in the field have - implemented (slightly) different approaches each of which misses - some part of a complete workflow description which is behind EBSD analyses: - Sample preparation, measurement, indexing, post-processing, paper... - - The here exemplified default plot do not so far apply relevant rotations - but takes the orientation values as they come from the origin and using - coloring them as they come. It is thus the scientists responsibility to - enter and check if the respective dataset is rotation-conventions-wise - consistent and fit for a particular task. - - Ideally, with all conventions defined it can be possible to develop - a converter which rotates the input data. This application definition - does not assume this and users should be aware of this limitation. - - The key point is that the conventions however are captured and this is - the most important step to the development of such a generic transcoder - for creating interoperable EBSD datasets. - - Currently the conventions remain in the mind or manual lab book of the - respective scientists or technicians instead of getting stored and - communicated with research papers that are written based on - specific dataset, i.e. database entries. - - The default gridded representation of the data should not be - misinterpreted as the only possible way how EBSD data and OIM - maps can be created! - - Indeed, the most general case is that patterns are collected for - scan points. The scan generator of an electron microscope is instructed - to steer the beam in such a way across the specimen surface that the - beam illuminates certain positions for a certain amount time (usually - equally-spaced and spending about the same amount of time at each - position). - - Therefore, scan positions can be due to such regular flight plans and - represent sampling on lines, line stacks, rectangular regions-of- - interests, but also could instruct spiral, random, or adaptive scans - instead of tessellations with square or hexagonal pixels. - - The majority of EBSD maps is though is reporting results for a regular - grid (square, hexagon). What matters though in terms of damage induced - by the electron beam and signal quality is the real electron dose - history, i.e. for how long the beam exposed which location of the - specimen. Especially when electron charging occurs (i.e. an excess - amount of charge accumulates due to e.g. poor conducting away of this - charge or an improper mounting, too high dose, etc. such details are - relevant. - - Specifically, the default visualization is an inverse pole-figure (IPF) - map with the usual RGB color coding. Different strategies and - normalization schemes are in use to define such color coding. - - Finally, we should mention that each ipf_map represents data for - scan points indexed as one phase. The alias/name of this phase should - be stored in phase_name, the phase_identifier give an ID which must - not be zero as this value is reserved for non-indexed / null model scan - points. - - - - Specifying which phase this IPF mapping visualizes. - - - - - Alias/name for the phase whose indexed scan points are displayed. - - - - - Which IPF definition computation according to backend. - - - - - - - Along which axis to project? Typically [0, 0, 1] is chosen. - - - - - - - - Bitdepth used for the RGB color model. Usually 8 bit. - - - - - - The tool/implementation used for creating the IPF color map from - the orientation data. Effectively, this program is the backend - which performs the computation of the inverse pole figure mappings - which can be for some use cases the parser. - Consider the explanations in the docstring of the ipf_mapID group. - - - - - - - - - The RGB image which represents the IPF map. - - - - - - - - - - RGB array, with resolution per fastest changing value - defined by bitdepth. - - - - - - - - - - IPF color-coded orientation mapping - - - - - - Calibrated center of mass of the pixel along the slow axis. - - - - - - - Label for the y axis - - - - - - - Calibrated center of mass of the pixel along the fast axis. - - - - - - - Label for the x axis - - - - - - - For each stereographic standard triangle (SST), i.e. a rendering of - the fundamental zone of the crystal-symmetry-reduced orientation space SO3, - it is possible to define a color model which assigns each point in - the fundamental zone a color. - Different mapping models are in use and implement (slightly) different - scaling relations. Differences are which base colors of the RGB - color model are placed in which extremal position of the SST - and where the white point is located. For further details see: - - * [G. Nolze et al.](https://doi.org/10.1107/S1600576716012942) - * Srikanth Patala and coworkers"'" work and of others. - - Details are implementation-specific and not standardized yet. - Given that the SST has a complicated geometry, it cannot yet be - visualized using tools like H5Web, which is why for now the em_om - parsers takes a rasterized image which is rendered by the backend - tool. - - - - - - - - - - RGB array, with resolution per fastest changing value defined by bitdepth. - - - - - - - - - - IPF color key in stereographic standard triangle (SST) - - - - - - Pixel coordinate along the slow axis. - - - - - - - Label for the y axis - - - - - - Pixel coordinate along the fast axis. - - - - - - - Label for the x axis - - - - - - - - - - This application definition also enables to describe a workflow where several - EBSD datasets are not only documented but also correlated based on time, - position (spatial), or both (spatiotemporal). - - Spatial correlations between repetitively characterized regions-of-interests - are typically correlated using image registration and alignment algorithms. - For this typically so-called landmarks are used. These can be grains with - a very large size or specific shape, i.e. grains which are qualitatively - different enough to be used as a guide how images are shifted relative to - one another. Other commonly used landmarks are fiducial marks which are - milled into the specimen surface using focus-ion beam milling and/or various - types of indentation methods. - - As far as the same physical region-of-interest is just measured several times, - the additional issue of the depth increment is not a concern. However, correct - assumptions for the depth increment, amount of material removed along the milling - direction is relevant for accurate and precise three-dimensional (serial-sectioning) - correlations. For these studies it can be tricky though to assume or estimate - useful depth increments. Different strategies have been proposed like - calibrations, wedged-shaped landmarks and computer simulation assisted - assumption making. - - Despite the use of landmarks, there are many practical issues which make the - processing of correlations imprecise and inaccurate. Among these are drift - and shift of the specimen, instabilities of the holder, the beam, irrespective - of the source of the drift, charging effects, here specifically causing local - image distortions and rotations which may require special processing algorithms - to reduce such imprecisions. - - Time correlations face all of the above-mentioned issues surplus the challenge - that specific experimental protocols have to be used to ensure the material state - is observed at specific physical time. The example of quasi in-situ characterization - of crystal growth phenomena, a common topic in engineering or modern catalysis research - makes it necessary to consider that e.g. the target value for the desired annealing - temperature is not just gauged based on macroscopic arguments but considers - that transient effects take place. Heating or quenching a sample might thus might - not have been executed under conditions in the interaction volume as they are - documented and/or assumed. - - These issue cause that correlations have an error margin as to how accurately - respective datasets were not only just synced based on the geometry of the - region-of-interests and the time markers but also to asssure which physical - conditions the specimen experienced over the course of the measurements. - - The fourth example of the em_om reference implementation explores the use of the - correlation group with a serial-sectioning datasets that was collected by the - classical Inconel 100 dataset collected by M. D. Uchic and colleagues - (M. Groeber M, Haley BK, Uchic MD, Dimiduk DM, Ghosh S 3d reconstruction and - characterization of polycrystalline microstructures using a fib-sem system data set. - Mater Charac 2006, 57 259–273. 10.1016/j.matchar.2006.01.019M). - - This dataset was specifically relevant in driving forward the implementation - of the DREAM.3D software. DREAM.3D is an open-source software project for - post-processing and reconstructing, i.e. correlating sets of orientation - microscopy data foremost spatially. One focus of the software is the - (post-)processing of EBSD datasets. Another cutting edge tool with similar - scope but a commercial solution by Bruker is QUBE which was developed by - P. Konijnenberg and coworkers. - - Conceptually, software like DREAM.3D supports users with creating linear - workflows of post-processing tasks. Workflows can be instructed via the - graphical user interface or via so-called pipeline processing via command line - calls. DREAM.3D is especially useful because its internal system documents all - input, output, and parameter of the processing steps. This makes DREAM.3D a - good candidate to interface with tools like em_om parser. Specifically, DREAM.3D - documents numerical results via a customized HDF5 file format called DREAM3D. - Workflow steps and settings are stored as nested dictionaries in JSON syntax - inside a supplementary JSON file or alongside the data in the DREAM3D file. - DREAM.3D has a few hundred algorithms implemented. These are called filters - in DREAM.3D terminology. - - Users configure a workflow which instructs DREAM.3D to send the data through - a chain of predefined and configured filters. Given that for each analysis - the filter is documented via its version tags surplus its parameter and setting - via a controlled vocabulary, interpreting the content of a DREAM3D HDF5 file - is possible in an automated manner using a parser. This makes DREAM.3D analyses - repeatable and self-descriptive. A key limitation though is that most frequently - the initial set of input data come from commercial files like ANG. - This missing link between the provenance of these input files, their associated - creation as electron microscope session, is also what NXem_ebsd solves. - - Nevertheless, as this can be solved with e.g. NXem_ebsd we are convinced that - the DREAM.3D and the em_om parser can work productively together to realize - RDMS-agnostic parsing of serial-section analyses. - - The internal documentation of the DREAM.3D workflow also simplifies the - provenance tracking represented by an instance of NXem_ebsd as not every - intermediate results has to be stored. Therefore, the fourth example - focuses on the key result obtained from DREAM.3D - the reconstructed - and aligned three-dimensional orientation map. - - Usually, this result is the starting point for further post-processing - and characterization of structural features. As here orientation microscopy - is insofar scale invariant using DREAM.3D, NXem_ebsd, and em_om should - be useful for different characterization methods, such as EBSD, Transmission - Kikuchi Diffraction (TKD), Automated Crystal Orientation Mapping (ACOM), - Nanobeam Electron Diffraction (using commercial systems like NanoMegas ASTAR) - or open-source implementations of these techniques (such as via pyxem/orix). - - The result of orientation microscopy methods are maps of local orientation - and thermodynamic phase (crystal structure) pieces of information. Virtually - all post-processing of such results for structural features includes again - a workflow of steps which are covered though by the NXms partner application - definition. The respective source of the data in an instance of NXms can - again be a link or reference to an instance of NXem_ebsd to complete the - chain of provenance. - - - - - - - - - - - - - - - - An overview of the entire reconstructed volume. For details about - what defines the image contrast inspect descriptor. - - - - Descriptor representing the image contrast. - - - - - - Container holding a default plot of the reconstructed volume. - - - - - - - - - - Descriptor values displaying the ROI. - - - - - - - - - - Signal - - - - - - Calibrated center of mass of the pixel along the slow axis. - - - - - - - Label for the z axis - - - - - - Calibrated center of mass of the pixel along the fast axis. - - - - - - - Label for the y axis - - - - - - Calibrated center of mass of the pixel along the fastest axis. - - - - - - - Label for the x axis - - - - - - - - Default inverse pole figure (IPF) plot of the data specific for each - phase. No ipf_mapID instances for non-indexed scan points as these are - by definition assigned the null phase with phase_identifier 0. - The same comments apply as to the two-dimensional representation. - - - - Specifying which phase this IPF mapping visualizes. - - - - - Alias/name for the phase whose indexed scan points are displayed. - - - - - Which IPF definition computation according to backend. - - - - - - Along which axis to project? Typically [0, 0, 1] is chosen. - - - - - - - - Bitdepth used for the RGB color model. Usually 8 bit. - - - - - The tool/implementation used for creating the IPF color map from - the orientation data. Effectively, this program is the backend - which performs the computation of the inverse pole figure mappings - which can be for some use cases the parser. - Consider the explanations in the docstring of the ipf_mapID group. - - - - - - - - - The RGB image which represents the IPF map. - - - - - - - - - - RGB array, with resolution per fastest changing value - defined by bitdepth. - - - - - - - - - - - IPF color-coded orientation mapping - - - - - - Calibrated center of mass of the pixel along the slow axis. - - - - - - - Label for the z axis - - - - - - - Calibrated center of mass of the pixel along the faster axis. - - - - - - - Label for the y axis - - - - - - - Calibrated center of mass of the pixel along the fastest axis. - - - - - - - Label for the x axis - - - - - - - Same comments as for the two-dimensional case apply. - - - - - - - - - - RGB array, with resolution per fastest changing value defined by bitdepth. - - - - - - - - - - IPF color key in stereographic standard triangle (SST) - - - - - - Pixel coordinate along the slow axis. - - - - - - - Label for the y axis - - - - - - Pixel coordinate along the fast axis. - - - - - - - Label for the x axis - - - - - - - - - - diff --git a/base_classes/NXevent_data_em.nxdl.xml b/base_classes/NXevent_data_em.nxdl.xml deleted file mode 100644 index f32978259a..0000000000 --- a/base_classes/NXevent_data_em.nxdl.xml +++ /dev/null @@ -1,226 +0,0 @@ - - - - - - Metadata and settings of an electron microscope for scans and images. - - The need for such a structuring of data is evident from the fact that - electron microscopes are dynamic. Oftentimes it suffices to calibrate the - instrument at the start of the session. Subsequently, data (images, spectra, etc.) - can be collected. Users may wish to take only a single scan or image and - complete their microscope session; however - - frequently users are working much longer at the microscope, recalibrate and take - multiple data items (scans, images, spectra). Each item comes with own detector - and eventually on-the-fly processing settings and calibrations. - - For the single data item use case one may argue that the need for an additional - grouping is redundant. Instead, the metadata could equally be stored inside - the respective groups of the top-level mandatory NXinstrument group. - On the flip side, even for a session with a single image, it would also not - harm to nest the data. - - In fact, oftentimes scientists feel that there is a need to store details - about eventual drift of the specimen in its holder (if such data is available) - or record changes to the lens excitations or apertures used and/or changed. - Although current microscopes are usually equipped with stabilization - systems for many of the individual components, it can still be useful - to store time-dependent data in detail. - - Another reason if not a need for having more finely granularizable options for - storing time-dependent data, is that over the course of a session one may - reconfigure the microscope. What is a reconfiguration? This could be the - change of an aperture mode because a scientist may first collect an image - with some aperture and then pick a different value and continue. - As the aperture affects the electron beam, it will affect the system. - - Let aside for a moment the technology and business models, an EM could be - monitored (and will likely become so more in the future) for streaming out - spatio-temporal details about its components, locations of (hardware components) and objects within the region-of-interest. Eventually external stimuli are applied - and the specimen repositioned. - - Some snapshot or integrated data from this stream are relevant for understanding - signal genesis and electron/ion-beam-sample interaction (paths). In such a generic - case it might be necessary to sync these streaming data with those intervals - in time when specific measurements are taken (spectra collected, - images taken, diffraction images indexed on-the-fly). - - Therefore, both the instrument and specimen should always be considered as dynamic. - Scientists often report or feel (difficult to quantify) observations that - microscopes *perform differently* across sessions, without sometimes being - able to identify clear root causes. Users of the instrument may consider - such conditions impractical, or *too poor* and thus either abort their session - or try to bring the microscope first into a state where conditions are considered - more stable, better, or of some whatever high enough quality to reuse - data collection. - - In all these cases it is practical to have a mechanism how time-dependent data - of the instrument state can be stored and documented in a interoperable way. - Where should these data be stored? With NeXus these data should not only be - stored in the respective instrument component groups of the top-level NXinstrument. - The reason is that this group should be reserved for as stable as possible - quantities which do not change over the session. Thereby we can avoid to store - repetitively that there is a certain gun or detector available but just store - the changes. This is exactly what the em_lab group is for inside NXevent_data_em. - - Ideally, NXevent_data_em are equipped with a start_time and end_time - to represent a time interval (remind the idea of the instrument state stream) - during which the scientist considered (or practically has to consider) - the microscope (especially ebeam and specimen) as stable enough. - - Arguably it is oftentimes tricky to specify a clear time interval when the - microscope is stable enough. Take for instance the acquisition of an image - or spectra stack. It is not fully possible (technically) to avoid that even - within a single image instabilities of the beam are faced and drift occurs. - Maybe in many cases this these instabilities are irrelevant but does this - warrant to create a data schema where either the microscope state can only - be stored very coarsely or one is forced to store it very finely? - - This is a question of how one wishes to granularize pieces of information. - A possible solution is to consider each probed position, i.e. point in time - when the beam was not blanked and thus when the beam illuminates a portion of - the material, i.e. the interaction volume, whose signal contributions are then - counted by the one or multiple detector(s) as per pixel- or per voxel signal - in the region-of-interest. - NXevent_data_em in combination with the NXem application definition - allows researchers to document this. Noteworty to mention is that we understand - that in many cases realizing such a fine temporal and logical granularization - and data collection is hard to achieve in practice. - - A frequently made choice, mainly for convenience, is that drift and scan distortions - are considered a feature or inaccuracy of the image and/or spectrum and thus - one de facto accepts that the microscope was not as stable as expected during - the acquisition of the image. We learn that the idea of a time interval - during the microscope session may be interpreted differently by different - users. Here we consider the choice to focus on images and spectra, and eventually - single position measurements as the smallest granularization level. - Which eventually may require to add optional NXprocess instances for respectively - collected data to describe the relevant distortions. Nevertheless, the distortions - are typically corrected for by numerical protocols. This fact warrants to - consider the distortion correction a computational workflow which can be - modelled as a chain of NXprocess instances each with own parameters. an own - A more detailed overview of such computational steps to cope with scan distortions - is available in the literature: - - * `C. Ophus et al. <https://dx.doi.org/10.1016/j.ultramic.2015.12.002>`_ - * `B. Berkels et al. <https://doi.org/10.1016/j.ultramic.2018.12.016>`_ - * `L. Jones et al. <https://link.springer.com/article/10.1186/s40679-015-0008-4>`_ - - For specific simulation purposes, mainly in an effort to digitally repeat - or simulate the experiment, it is tempting to consider dynamics of the - instrument, implemented as time-dependent functional descriptions of - e.g. lens excitations, beam shape functions, trajectories of groups of - electrons, or detector noise models. - - For now the preferred strategy to handle these cases is through - customizations of the specific fields within NXevent_data_em instances. - - Another alternative could be to sample finer, eventually dissimilarly along - the time axi; however this may cause situations where an NXevent_data_em - instance does not contain specific measurements (i.e. images, spectra of - scientific relevance). - - In this case one should better go for a customized application definition - with a functional property description inside members (fields or groups) - in NXevent_data_em instances; or resort to a specific offspring application - definition of NXem which documents metadata for tracking explicitly electrons - (with ray-tracing based descriptors/computational geometry descriptors) - or tracking of wave bundles. - - This perspective on much more subtle time-dependent considerations of electron - microscopy can be advantageous also for storing details of time-dependent - additional components that are coupled to and/or synced with a microscope. - - Examples include cutting-edge experiments where the electron beam gets - coupled/excited by e.g. lasers. In this case, the laser unit should be - registered under the top-level NXinstrument section. Its spatio-temporal - details could be stored inside respective additional groups of the NXinstrument - though inside instances of here detailed NXevent_data_em. - - - - ISO 8601 time code with local time zone offset to UTC information included - when the snapshot time interval started. If the user wishes to specify an - interval of time that the snapshot should represent during which the instrument - was stable and configured using specific settings and calibrations, - the start_time is the start (left bound of the time interval) while - the end_time specifies the end (right bound) of the time interval. - - - - - ISO 8601 time code with local time zone offset to UTC information included - when the snapshot time interval ended. - - - - - Reference to a specific state and setting of the microscope. - - - - - Which specific event/measurement type. Examples are: - - * In-lens/backscattered electron, usually has quadrants - * Secondary_electron, image, topography, fractography, overview images - * Backscattered_electron, image, Z or channeling contrast (ECCI) - * Bright_field, image, TEM - * Dark_field, image, crystal defects - * Annular dark field, image (medium- or high-angle), TEM - * Diffraction, image, TEM, or a comparable technique in the SEM - * Kikuchi, image, SEM EBSD and TEM diffraction - * X-ray spectra (point, line, surface, volume), composition EDS/EDX(S) - * Electron energy loss spectra for points, lines, surfaces, TEM - * Auger, spectrum, (low Z contrast element composition) - * Cathodoluminescence (optical spectra) - * Ronchigram, image, alignment utility specifically in TEM - * Chamber, e.g. TV camera inside the chamber, education purposes. - - This field may also be used for storing additional information about the event. - - - - - - - - - diff --git a/base_classes/NXibeam_column.nxdl.xml b/base_classes/NXibeam_column.nxdl.xml deleted file mode 100644 index eea601c4e8..0000000000 --- a/base_classes/NXibeam_column.nxdl.xml +++ /dev/null @@ -1,144 +0,0 @@ - - - - - - - Container for components of a focused-ion-beam (FIB) system. - - FIB capabilities turn especially scanning electron microscopes - into specimen preparation labs. FIB is a material preparation - technique whereby portions of the sample are illuminated with a - focused ion beam with controlled intensity intense enough and with - sufficient ion momentum to remove material in a controllable manner. - - The fact that an electron microscope with FIB capabilities has needs a - second gun with own relevant control circuits, focusing lenses, and - other components, warrants an own base class to group these components - and distinguish them from the lenses and components for creating and - shaping the electron beam. - - For more details about the relevant physics and application examples - consult the literature, for example: - - * `L. A. Giannuzzi et al. <https://doi.org/10.1007/b101190>`_ - * `E. I. Preiß et al. <https://link.springer.com/content/pdf/10.1557/s43578-020-00045-w.pdf>`_ - * `J. F. Ziegler et al. <https://www.sciencedirect.com/science/article/pii/S0168583X10001862>`_ - * `J. Lili <https://www.osti.gov/servlets/purl/924801>`_ - - - - - The source which creates the ion beam. - - - - Given name/alias for the ion gun. - - - - - Emitter type used to create the ion beam. - - If the emitter type is other, give further - details in the description field. - - - - - - - - - - - Ideally, a (globally) unique persistent identifier, link, - or text to a resource which gives further details. - - - - - Which ionized elements or molecular ions form the beam. - Examples are gallium, helium, neon, argon, krypton, - or xenon, O2+. - - - - - - Average/nominal brightness - - - - - - Charge current - - - - - Ion acceleration voltage upon source exit and entering the vacuum flight path. - - - - - - Affine transformation which detail the arrangement in the microscope relative to - the optical axis and beam path. - - - - - - - - - - - Individual characterization results for the position, shape, - and characteristics of the ion beam. - - NXtransformations should be used to specify the location or position - at which details about the ion beam are probed. - - - - diff --git a/base_classes/NXion.nxdl.xml b/base_classes/NXion.nxdl.xml deleted file mode 100644 index 99a19f2e3c..0000000000 --- a/base_classes/NXion.nxdl.xml +++ /dev/null @@ -1,168 +0,0 @@ - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - Maximum number of atoms/isotopes allowed per (molecular) ion (fragment). - - - - - Number of mass-to-charge-state-ratio range intervals for ion type. - - - - - Set of atoms of a molecular ion or fragment in e.g. ToF mass spectrometry. - - - - A unique identifier whereby such an ion can be referred to - via the service offered as described in identifier_type. - - - - - How can the identifier be resolved? - - - - - - - - Ion type (ion species) identifier. The identifier zero - is reserved for the special unknown ion type. - - - - - A vector of isotope hash values. - These values have to be stored in an array, sorted in decreasing order. - The array is filled with zero hash values indicating unused places. - The individual hash values are built with the following hash function: - - The hash value :math:`H` is :math:`H = Z + N*256` with :math:`Z` - the number of protons and :math:`N` the number of neutrons - of each isotope respectively. - - Z and N have to be 8-bit unsigned integers. - For the rationale behind this `M. Kühbach et al. (2021) <https://doi.org/10.1017/S1431927621012241>`_ - - - - - - - - - A supplementary row vector which decodes the isotope_vector into - a human-readable matrix of nuclids with the following formatting: - - The first row specifies the isotope mass number, i.e. using the hashvalues - from the isotope_vector this is :math:`Z + N`. As an example for a - carbon-14 isotope the number is 14. - The second row specifies the number of protons :math:`Z`, e.g. 6 for the - carbon-14 example. This row matrix is thus a mapping the notation of - using superscribed isotope mass and subscripted number of protons to - identify isotopes. - Unused places filling up to n_ivecmax need to be filled with zero. - - - - - - - - - Color code used for visualizing such ions. - - - - - Assumed volume of the ion. - - In atom probe microscopy this field can be used to store the reconstructed - volume per ion (average) which is typically stored in range files and will - be used when building a tomographic reconstruction of an atom probe - dataset. - - - - - Charge of the ion. - - - - - Signed charge state of the ion in multiples of electron charge. - - Only positive values will be measured in atom probe microscopy as the - ions are accelerated by a negatively signed bias electric field. - In the case that the charge state is not explicitly recoverable, - the value should be set to zero. - - In atom probe microscopy this is for example the case when using - classical range file formats like RNG, RRNG for atom probe data. - These file formats do not document the charge state explicitly. - They report the number of atoms of each element per molecular ion - surplus the mass-to-charge-state-ratio interval. - With this it is possible to recover the charge state only for - specific molecular ions as the accumulated mass of the molecular ion - is defined by the isotopes, which without knowing the charge leads - to an underconstrained problem. - Details on ranging can be found in the literature: `M. K. Miller <https://doi.org/10.1002/sia.1719>`_ - - - - - Human-readable ion type name (e.g. Al +++) - The string should consists of ASCII UTF-8 characters, - ideally using LaTeX notation to specify the isotopes, ions, and charge - state. Examples are 12C + or Al +++. - Although this name may be human-readable and intuitive, parsing such - names becomes impractical for more complicated cases. Therefore, for the - field of atom probe microscopy the isotope_vector should be the - preferred machine-readable format to use. - - - - - Associated lower (mqmin) and upper (mqmax) bounds of - mass-to-charge-state ratio interval(s) [mqmin, mqmax] - (boundaries included) for which the respective ion is one to be labelled - with ion_identifier. The field is primarily of interest to document the - result of indexing a ToF/mass spectrum. - - - - - - - - diff --git a/base_classes/NXlens_em.nxdl.xml b/base_classes/NXlens_em.nxdl.xml deleted file mode 100644 index 925aa35aa1..0000000000 --- a/base_classes/NXlens_em.nxdl.xml +++ /dev/null @@ -1,109 +0,0 @@ - - - - - - Description of an electro-magnetic lens or a compound lens. - - For NXtransformations the origin of the coordinate system is placed - in the center of the lens - (its polepiece, pinhole, or another point of reference). - The origin should be specified in the NXtransformations. - - For details of electro-magnetic lenses in the literature see e.g. `L. Reimer <https://doi.org/10.1007/978-3-540-38967-5>`_ - - - - Qualitative type of lens with respect to the number of pole pieces. - - - - - - - - - - - - Given name, alias, colloquial, or short name for the lens. - For manufacturer names and identifiers use respective manufacturer fields. - - - - - Name of the manufacturer who built/constructed the lens. - - - - - - Hardware name, hash identifier, or serial number that was given by the - manufacturer to identify the lens. - - - - - Ideally an identifier, persistent link, or free text which gives further details - about the lens. - - - - - Excitation voltage of the lens. For dipoles it is a single number. For higher - orders, it is an array. - - - - - Excitation current of the lens. For dipoles it is a single number. For higher - orders, it is an array. - - - - - This field should be used when the exact voltage or current of the lens is not directly controllable - as the control software of the microscope does not enable users/or is was not configured to enable - the user to retrieve these values. In this case this field should be used to specify the value as - read from the control software. Although consumers of the application definition should not expect - this value to represent the exact physical voltage or excitation, it is still useful to know though - as it allows other users to reuse this lens setting, which, provided a properly working instrument - and software should bring the lenses into a similar state. - - - - - Specifies the position of the lens by pointing to the last transformation in the - transformation chain in the NXtransformations group. - - - - - Collection of axis-based translations and rotations to describe the - location and geometry of the lens as a component in the instrument. - Typically, the components of a system should all be related relative to - each other and only one component should relate to the reference - coordinate system. - - - diff --git a/base_classes/NXoptical_system_em.nxdl.xml b/base_classes/NXoptical_system_em.nxdl.xml deleted file mode 100644 index 0f753001ed..0000000000 --- a/base_classes/NXoptical_system_em.nxdl.xml +++ /dev/null @@ -1,83 +0,0 @@ - - - - - - A container for qualifying an electron optical system. - - - - - Citing the JEOL TEM glossary this is *an effective distance from a specimen - to a plane where an observed diffraction pattern is formed*. - - - - - The factor of enlargement of the apparent size, not physical size, of an object. - - - - - The defocus aberration constant oftentimes taken as the C_1_0 which - is described in more details in NXaberration. - - - - - Citing the JEOL TEM glosssary this is the value *when a cone shaped, - convergent electron beam illuminates a specimen, the semi-angle of the cone - is termed convergence angle.* - - - - - The extent of the observable parts of the specimen given the current - magnification and other settings of the instrument. - - - - - Citing `Globalsino <https://www.globalsino.com/EM/page4586.html>`_ this is - *a distance between the specimen and the lower pole piece in SEM system*. - - - - - Beam current as measured relevant for the illumination of the specimen. - Users should specify further details like how the beam current was measured - using the beam_current_description field. - - - - - Specify further details how the beam current was measured or estimated. - - - - diff --git a/base_classes/NXregistration.nxdl.xml b/base_classes/NXregistration.nxdl.xml deleted file mode 100644 index a33ba3b502..0000000000 --- a/base_classes/NXregistration.nxdl.xml +++ /dev/null @@ -1,55 +0,0 @@ - - - - - - Describes image registration procedures. - - - - Has the registration been applied? - - - - - Indicates the name of the last operation applied in the NXprocess sequence. - - - - - Specifies the position by pointing to the last transformation in the - transformation chain in the NXtransformations group. - - - - - To describe the operations of image registration (combinations of rigid - translations and rotations) - - - - - Description of the procedures employed. - - - diff --git a/base_classes/NXscanbox_em.nxdl.xml b/base_classes/NXscanbox_em.nxdl.xml deleted file mode 100644 index c95f62357d..0000000000 --- a/base_classes/NXscanbox_em.nxdl.xml +++ /dev/null @@ -1,47 +0,0 @@ - - - - - - Scan box and coils which deflect an electron beam in a controlled manner. - - In electron microscopy, the scan box is instructed by the microscope - control software. This component directs the probe to controlled - locations according to a scan scheme and plan. - - - - - - - - - - - - - - diff --git a/base_classes/NXspectrum_set.nxdl.xml b/base_classes/NXspectrum_set.nxdl.xml deleted file mode 100644 index 9ca3aab349..0000000000 --- a/base_classes/NXspectrum_set.nxdl.xml +++ /dev/null @@ -1,162 +0,0 @@ - - - - - - - - - Number of pixel in the slow direction. - - - - - Number of pixel in the fast direction. - - - - - Number of energy bins. - - - - - Container for reporting a set of spectra. - - - - Details how spectra were processed from the detector readings. - - - - Resolvable data artifact (e.g. filename) from which the all values in - the NXdata instances in this NXspectrum_set were loaded during parsing. - - - - An at least as strong as SHA256 hashvalue of the data artifact which - source represents digitally to support provenance tracking. - - - - - - Imaging (data collection) mode of the instrument during acquisition - of the data in this NXspectrum_set instance. - - - - - Link or name of an NXdetector instance with which the data were collected. - - - - - - - - Collected spectra for all pixels of a rectangular region-of-interest. - This representation supports rectangular scan pattern. - - - - - - - - - - Counts - - - - - - - - - - - Coordinate along y direction - - - - - - - - - - Coordinate along x direction - - - - - - - - - - Energy - - - - - - - - Accumulated counts over all pixels of the region-of-interest. - This representation supports rectangular scan pattern. - - - - - - - - Counts - - - - - - - - - - - Energy - - - - - diff --git a/base_classes/NXstage_lab.nxdl.xml b/base_classes/NXstage_lab.nxdl.xml deleted file mode 100644 index 7b916d2728..0000000000 --- a/base_classes/NXstage_lab.nxdl.xml +++ /dev/null @@ -1,173 +0,0 @@ - - - - - - A stage lab can be used to hold, align, orient, and prepare a specimen. - - Modern stages are multi-functional devices. Many of which offer a controlled - environment around (a part) of the specimen. Stages enable experimentalists - to apply stimuli. A stage_lab is a multi-purpose/-functional tools which - can have multiple actuators, sensors, and other components. - - With such stages comes the need for storing various (meta)data - that are generated while manipulating the sample. - - Modern stages realize a hierarchy of components: For example the specimen - might be mounted on a multi-axial tilt rotation holder. This holder is - fixed in the support unit which connects the holder to the rest of the - microscope. - - In other examples, taken from atom probe microscopy, researchers may work - with wire samples which are clipped into a larger fixing unit for - convenience and enable for a more careful specimen handling. - This fixture unit is known in atom probe jargon as a stub. - Stubs in turn are positioned onto pucks. - Pucks are then loaded onto carousels. - A carousel is a carrier unit with which eventually entire sets of specimens - can be moved in between parts of the microscope. - - An NXstage_lab instance reflects this hierarchical design. The stage is the - root of the hierarchy. A stage carries the holder. - In the case that it is not practical to distinguish these two layers, - the holder should be given preference. - - Some examples for stage_labs in applications: - - * A nanoparticle on a copper grid. The copper grid is the holder. - The grid itself is fixed to the stage. - * An atom probe specimen fixed in a stub. In this case the stub can be - considered the holder, while the cryostat temperature control unit is - a component of the stage. - * Samples with arrays of specimens, like a microtip on a microtip array - is an example of a three-layer hierarchy commonly employed for - efficient sequential processing of atom probe experiments. - * With one entry of an application definition only one microtip should be - described. Therefore, the microtip is the specimen, - the array is the holder and the remaining mounting unit - that is attached to the cryo-controller is the stage. - * For in-situ experiments with e.g. chips with read-out electronics - as actuators, the chips are again placed in a larger unit. - * Other examples are (quasi) in-situ experiments where experimentalists - anneal or deform the specimen via e.g. in-situ tensile testing machines - which are mounted on the specimen holder. - - To cover for an as flexible design of complex stages, users should nest - multiple instances of NXstage_lab objects according to their needs to reflect - the differences between what they consider as the holder and what - they consider is the stage. - - Instances should be named with integers starting from 1 as the top level unit. - In the microtip example stage_lab_1 for the stage, stage_lab_2 for the holder - (microtip array), stage_lab_3 for the microtip specimen, respectively. - The depends_on keyword should be used with relative or absolute naming inside - the file to specify how different stage_lab instances build a hierarchy - if this is not obvious from numbered identifiers like the stage_lab_1 to - stage_lab 3 example. The lower it is the number the higher it is the - rank in the hierarchy. - - For specific details and inspiration about stages in electron microscopes: - - * `Holders with multiple axes <https://www.nanotechnik.com/e5as.html>`_ - * `Chip-based designs <https://www.protochips.com/products/fusion/fusion-select-components/>`_ - * `Further chip-based designs <https://www.nanoprobetech.com/about>`_ - * `Stages in transmission electron microscopy <https://doi.org/10.1007/978-3-662-14824-2>`_ (page 103, table 4.2) - * `Further stages in transmission electron microscopy <https://doi.org/10.1007/978-1-4757-2519-3>`_ (page 124ff) - * `Specimens in atom probe <https://doi.org/10.1007/978-1-4614-8721-0>`_ (page 47ff) - * `Exemplar micro-manipulators <https://nano.oxinst.com/products/omniprobe/omniprobe-200>`_ - - - - Principal design of the stage. - - Exemplar terms could be side_entry, top_entry, - single_tilt, quick_change, multiple_specimen, - bulk_specimen, double_tilt, tilt_rotate, - heating_chip, atmosphere_chip, - electrical_biasing_chip, liquid_cell_chip - - - - - Given name/alias for the components making the stage. - - - - - - Ideally, a (globally) unique persistent identifier, link, - or text to a resource which gives further details. - - - - - Should be defined by the application definition. - - - - - Should be defined by the application definition. - - - - - Should be defined by the application definition. - - - - - Should be defined by the application definition. - - - - - - - - Voltage applied to the stage to decelerate electrons. - - - - - - The rotation, tilt and position of stage components can be specified - either via NXtransformations or via the tilt_1, tilt_2, rotation, - and position fields. - - - - - diff --git a/contributed_definitions/NXaberration_model.nxdl.xml b/contributed_definitions/NXaberration_model.nxdl.xml index db06948bd2..c340fc2aed 100644 --- a/contributed_definitions/NXaberration_model.nxdl.xml +++ b/contributed_definitions/NXaberration_model.nxdl.xml @@ -2,9 +2,9 @@ + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The dimensionality, which has to be at least 2. + + + + + The cardinality of the set, i.e. the number of ellipses, or ellipsoids. + + + + + Computational geometry description of a set of ellipsoids in Euclidean space. + + Individual ellipsoids can have different half axes. + + + + + + Integer which specifies the first index to be used for distinguishing + identifiers for ellipsoids. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish ellipsoids for explicit indexing. + + + + + + + + Geometric center of the ellipsoids. This can be the center of mass. + Dimensionality and cardinality of the point and ellipsoid set have to match. + The identifier_offset and identifier field of NXcg_point_set do not need + to be used as they should be same as the identifier_offset and the + identifier for the ellipsoids. + + + + + + + + + If all ellipsoids in the set have the same half-axes. + + + + + + + + In the case that ellipsoids have different radii use this field + instead of half_axes_radius. + + + + + + + + + Reference to or definition of a coordinate system with + which the positions and directions are interpretable. + + + + + + Are the ellipsoids closed or hollow? + + + + + + + + + + + + + + + + + + + Direction unit vector which points along the largest half-axes. + + + + + + + diff --git a/contributed_definitions/NXcg_hexahedron_set.nxdl.xml b/contributed_definitions/NXcg_hexahedron_set.nxdl.xml new file mode 100644 index 0000000000..96c2d7123c --- /dev/null +++ b/contributed_definitions/NXcg_hexahedron_set.nxdl.xml @@ -0,0 +1,239 @@ + + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The cardinality of the set, i.e. the number of hexahedra. + + + + + Computational geometry description of a set of hexahedra in Euclidean space. + + The hexahedra do not have to be connected, can have different size, + can intersect, and be rotated. + This class can also be used to describe cuboids or cubes, axis-aligned or not. + The class represents different access and description levels to offer both + applied scientists and computational geometry experts to use the same + base class but rather their specific view on the data: + + * Most simple many experimentalists wish to communicate dimensions/the size + of specimens. In this case the alignment with axes is not relevant as + eventually the only relevant information to convey is the volume of the + specimen. + * In many cases, take for instance an experiment where a specimen was taken + from a specifically deformed piece of material, e.g. cold-rolled, + channel-die deformed, the orientation of the specimen edges with the + experiment coordinate system can be of very high relevance. + Examples include to know which specimen edge is parallel to the rolling, + the transverse, or the normal direction. + * Sufficient to pinpoint the sample and laboratory/experiment coordinate + system, the above-mentioned descriptions are not detailed enough though + to create a CAD model of the specimen. + * Therefore, groups and fields for an additional, computational-geometry- + based view of the hexahedra is offered which serve different computational + tasks: storage-oriented simple views or detailed topological/graph-based + descriptions. + + Hexahedra are important geometrical primitives, which are among the most + frequently used elements in finite element meshing/modeling. + + Hexahedra have to be non-degenerated, closed, and built of polygons + which are not self-intersecting. + + The term hexahedra will be used throughout this base class but includes + the especially in engineering and more commonly used special cases, + cuboid, cube, box, axis-aligned bounding box (AABB), optimal bounding + box (OBB). + + An axis-aligned bounding box is a common data object in + computational science and codes to represent a cuboid whose edges + are aligned with a coordinate system. As a part of binary trees these + are important data objects for time as well as space efficient queries + of geometric primitives in techniques like kd-trees. + + An optimal bounding box is a common data object which provides the best + tight fitting box about an arbitrary object. In general such boxes are + rotated. Exact and substantially faster in practice approximate algorithms + exist for computing optimal or near optimal bounding boxes for point sets. + + + + + + + + + + + A qualitative description of each hexahedron/cuboid/cube/box. + + + + + + + + + Qualifier how one edge is longer than all other edges of the hexahedra. + Often the term length is associated with one edge being parallel to + an axis of the coordinate system. + + + + + + + + Qualifier often used to describe the length of an edge within + a specific coordinate system. + + + + + + + + Qualifier often used to describe the length of an edge within + a specific coordinate system. + + + + + + + + Position of the geometric center, which often is but not necessarily + has to be the center_of_mass of the hexahedrally-shaped sample/sample part. + + + + + + + + + + + + + + Total area (of all six faces) of each hexahedron. + + + + + + + + Area of each of the six faces of each hexahedron. + + + + + + + + + Specifies if the hexahedra represent cuboids or cubes eventually rotated + ones but at least not too exotic six-faced polyhedra. + + + + + + + + Only to be used if is_box is present. In this case, this field describes + whether hexahedra are boxes whose primary edges are parallel to the + axes of the Cartesian coordinate system. + + + + + + + + Reference to or definition of a coordinate system with + which the qualifiers and mesh data are interpretable. + + + + + Integer which specifies the first index to be used for distinguishing + hexahedra. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish hexahedra for explicit indexing. + + + + + + + + + + + + + + + A simple approach to describe the entire set of hexahedra when the + main intention is to store the shape of the hexahedra for visualization. + + + + + + Disentangled representations of the mesh of specific hexahedra. + + + + + + Disentangled representation of the planar graph that each hexahedron + represents. Such a description simplifies topological processing + or analyses of mesh primitive operations and neighborhood queries. + + + diff --git a/contributed_definitions/NXcg_parallelogram_set.nxdl.xml b/contributed_definitions/NXcg_parallelogram_set.nxdl.xml new file mode 100644 index 0000000000..ca4a569845 --- /dev/null +++ b/contributed_definitions/NXcg_parallelogram_set.nxdl.xml @@ -0,0 +1,183 @@ + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The cardinality of the set, i.e. the number of parallelograms. + + + + + Computational geometry description of a set of parallelograms in Euclidean space. + + The parallelograms do not have to be connected, can have different size, + can intersect, and be rotated. + This class can also be used to describe rectangles or squares, axis-aligned or not. + The class represents different access and description levels to offer both + applied scientists and computational geometry experts to use the same + base class but rather their specific view on the data: + + * Most simple many experimentalists wish to communicate dimensions/the size + of e.g. a region of interest in the 2D plane. In this case the alignment + with axes is not relevant as eventually relevant is only the area of the ROI. + * In other cases the extent of the parallelogram is relevant though. + * Finally in CAD models we would like to specify the polygon + which is parallelogram represents. + + Parallelograms are important geometrical primitives. Not so much because of their + uses in nowadays, thanks to improvements in computing power, not so frequently + any longer performed 2D simulation. Many scanning experiments probe though + parallelogram-shaped ROIs on the surface of samples. + + Parallelograms have to be non-degenerated, closed, and built of polygons + which are not self-intersecting. + + The term parallelogram will be used throughout this base class but includes + the especially in engineering and more commonly used special cases, + rectangle, square, 2D box, axis-aligned bounding box (AABB), or + optimal bounding box (OBB) but here the analogous 2D cases. + + An axis-aligned bounding box is a common data object in computational science + and codes to represent a rectangle whose edges are aligned with the axes + of a coordinate system. As a part of binary trees these are important data + objects for time- as well as space-efficient queries + of geometric primitives in techniques like kd-trees. + + An optimal bounding box is a common data object which provides the best + tight fitting box about an arbitrary object. In general such boxes are + rotated. Other than in 3D dimensions the rotation calipher method offers + a rigorous approach to compute optimal bounding boxes in 2D. + + + + + + + + + + + A qualitative description of each parallelogram/rectangle/square/box. + + + + + + + + + Qualifier how one edge is longer than all the other edge of the parallelogam. + Often the term length is associated with one edge being parallel to + an axis of the coordinate system. + + + + + + + + Qualifier often used to describe the length of an edge within + a specific coordinate system. + + + + + + + + Position of the geometric center, which often is but not necessarily + has to be the center_of_mass of the parallelogram. + + + + + + + + + + + + + + Only to be used if is_box is present. In this case, this field describes + whether parallelograms are rectangles/squares whose primary edges + are parallel to the axes of the Cartesian coordinate system. + + + + + + + + Reference to or definition of a coordinate system with + which the qualifiers and mesh data are interpretable. + + + + + Integer which specifies the first index to be used for distinguishing + parallelograms. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish parallelograms for explicit indexing. + + + + + + + + + + + + + + + A simple approach to describe the entire set of parallelograms when the + main intention is to store the shape of the parallelograms for visualization. + + + + + + Disentangled representations of the mesh of specific parallelograms. + + + + diff --git a/contributed_definitions/NXcg_point_set.nxdl.xml b/contributed_definitions/NXcg_point_set.nxdl.xml new file mode 100644 index 0000000000..e5c351839c --- /dev/null +++ b/contributed_definitions/NXcg_point_set.nxdl.xml @@ -0,0 +1,98 @@ + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The dimensionality, which has to be at least 1. + + + + + The cardinality of the set, i.e. the number of points. + + + + + Computational geometry description of a set of points in Euclidean space. + + The relevant coordinate system should be referred to in the NXtransformations + instance. Points may have an associated time value; however users are advised + to store time data of point sets rather as instances of time events, where + for each point in time there is an NXcg_point_set instance which specifies the + points locations. This is a frequent situation in experiments and computer + simulations, where positions of points are taken at the same point in time; + and therefore an additional time array would demand to store redundant pieces + of information for each point. + + + + + + Integer which specifies the first index to be used for distinguishing + identifiers for points. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish points for explicit indexing. + + + + + + + + The array of point coordinates. + + + + + + + + + The optional array of time for each vertex. + + + + + + + + Reference to or definition of a coordinate system with + which the positions and directions are interpretable. + + + diff --git a/contributed_definitions/NXcg_polygon_set.nxdl.xml b/contributed_definitions/NXcg_polygon_set.nxdl.xml new file mode 100644 index 0000000000..e90dd652f4 --- /dev/null +++ b/contributed_definitions/NXcg_polygon_set.nxdl.xml @@ -0,0 +1,225 @@ + + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The dimensionality, which has to be either 2 or 3. + + + + + The cardinality of the set, i.e. the number of polygons. + + + + + + The total number of vertices when visiting every polygon. + + + + + + Computational geometry description of a set of polygons in Euclidean space. + + Polygons are related are specialized polylines: + + * A polygon is a geometric primitive that is bounded by a closed polyline + * All vertices of this polyline lay in the d-1 dimensional plane. + whereas vertices of a polyline do not necessarily lay on a plane. + * A polygon has at least three vertices. + + Each polygon is built from a sequence of vertices (points with identifiers). + The members of a set of polygons may have a different number of vertices. + Sometimes a collection/set of polygons is referred to as a soup of polygons. + + As three-dimensional objects, a set of polygons can be used to define the + hull of what is effectively a polyhedron; however users are advised to use + the specific NXcg_polyhedron_set base class if they wish to describe closed + polyhedra. Even more general complexes can be thought, for instance + piecewise-linear complexes, as these can have holes though, polyhedra without + holes are one subclass of such complexes, users should rather design an own + base class e.g. NXcg_polytope_set to describe such even more + complex primitives. + + + + + + + + + + + + Integer which specifies the first index to be used for distinguishing + polygons. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish polygons for explicit indexing. + + + + + + + + + A simple approach to describe the entire set of polygons when the + main intention is to store the shape of the polygons for visualization. + + + + + + + + + + + + + + + The accumulated length of the polygon edge. + + + + + + + + Array of interior angles. There are many values per polygon as number_of_vertices. + The angle is the angle at the specific vertex, i.e. between the adjoining + edges of the vertex according to the sequence in the polygons array. + Usually, the winding_order field is required to interpret the value. + + + + + + + + Curvature type: + + * 0 - unspecified, + * 1 - convex, + * 2 - concave + + + + + + + + The center of mass of each polygon. + + + + + + + + + Axis-aligned or (approximate) (optimal) bounding boxes to each polygon. + + + + diff --git a/contributed_definitions/NXcg_polyhedron_set.nxdl.xml b/contributed_definitions/NXcg_polyhedron_set.nxdl.xml new file mode 100644 index 0000000000..e3a6e99fe4 --- /dev/null +++ b/contributed_definitions/NXcg_polyhedron_set.nxdl.xml @@ -0,0 +1,194 @@ + + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The cardinality of the set, i.e. the number of polyhedra. + + + + + The total number of edges for all polyhedra. + + + + + The total number of faces for all polyhedra. + + + + + Computational geometry description of a polyhedra in Euclidean space. + + Polyhedra, also so-called cells (especially in the convex of tessellations), + here described have to be all non-degenerated, closed, built of and thus + built out of not-self-intersecting polygon meshes. Polyhedra may make contact, + so that this base class can be used for a future description of tessellations. + + For more complicated manifolds and especially for polyhedra with holes, users + are advised to check if their particular needs are described by creating + (eventually customized) instances of an NXcg_polygon_set, which can be + extended for the description of piecewise-linear complexes. + + + + + + + + + + + Interior volume + + + + + + + + Position of the geometric center, which often is but not necessarily + has to be the center_of_mass of the polyhedra. + + + + + + + + + Total surface area as the sum of all faces. + + + + + + + + The number of faces for each polyhedron. Faces of adjoining polyhedra + are counted for each polyhedron. This field can be used to interpret + the array/field with the individual area values for each face. + + + + + + + + Area of each of the four triangular faces of each tetrahedron. + + + + + + + + The number of edges for each polyhedron. Edges of adjoining polyhedra + are counterd for each polyhedron. This field can be used to interpret + the array/field with the individual length for each edge. + + + + + Length of each edge of each tetrahedron. + + + + + + + + Reference to or definition of a coordinate system with + which the qualifiers and mesh data are interpretable. + + + + + + Integer which specifies the first index to be used for distinguishing + polyhedra. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish polyhedra for explicit indexing. + + + + + + + + + + + + + A simple approach to describe the entire set of polyhedra when the + main intention is to store the shape of the polyhedra for visualization. + + + + + + Disentangled representations of the mesh of specific polyhedron. + + + + + + Disentangled representation of the planar graph that each polyhedron + represents. Such a description simplifies topological processing + or analyses of mesh primitive operations and neighborhood queries. + + + + diff --git a/contributed_definitions/NXcg_primitive_set.nxdl.xml b/contributed_definitions/NXcg_primitive_set.nxdl.xml new file mode 100644 index 0000000000..ac451bdc66 --- /dev/null +++ b/contributed_definitions/NXcg_primitive_set.nxdl.xml @@ -0,0 +1,212 @@ + + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The dimensionality of the space. + + + + + The cardinality of the set, i.e. the number of members. + + + + + Computational geometry description of a set of primitives in Euclidean space. + + Primitives must neither be degenerated nor self-intersect. + Individual primitives can differ in their properties (e.g. size, shape, rotation). + + + + + Hint to help resolve in which Euclidean coordinate system field values + like center or orientation are defined. + + + + + The dimensionality of the primitive set. + + + + + + + + + + The cardinality of the primitive set. + + + + + Integer offset whereby the identifier of the first member + of the set differs from zero. + + Identifiers can be defined either implicitly or explicitly. + For implicit indexing identifiers are defined on the interval + :math:`[identifier_offset, identifier_offset + c - 1]`. + + Therefore, implicit identifier are completely defined by the value of + identifier_offset and cardinality. For example if identifier run from + -2 to 3 the value for identifier_offset is -2. + + For explicit indexing the field identifier has to be used. + Fortran-/Matlab- and C-/Python-style indexing have specific implicit + identifier conventions where identifier_offset is 1 and 0 respectively. + + + + + Identifier of each member for explicit indexing. + + + + + + + + The center of mass position of each primitive. + + + + + + + + + + True if the center is a center of mass. + + + + + + + + A qualitative description of the shape of each primitive. + + + + + + + + + Qualifier for the length of characteristic features of the primitive. + + Often the term length is associated with the assumption that one + edge is parallel to an axis of the coordinate system. + + + + + + + + Qualifier often used to describe the length of one characteristic edge + within the coordinate system. + + + + + + + + True if primitive is closed such that it has properties like area or volume. + + + + + + + + Volume of each primitive. + + Set to NaN if does not apply for primitives for which is_closed is False. + + + + + + + + Alias for surface_area of each primitive. + + Set to NaN if does not apply for primitives for which is_closed is False. + + + + + + + + Direction unit vector which points along the + longest principal axis of each primitive. + + Use the depends_on attribute to specify in which coordinate system + these direction unit vectors are defined. + + + + + + + + + + + diff --git a/contributed_definitions/NXcg_sphere_set.nxdl.xml b/contributed_definitions/NXcg_sphere_set.nxdl.xml new file mode 100644 index 0000000000..e50192cf85 --- /dev/null +++ b/contributed_definitions/NXcg_sphere_set.nxdl.xml @@ -0,0 +1,121 @@ + + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The dimensionality, which has to be at least 2. + + + + + The cardinality of the set, i.e. the number of circles or spheres. + + + + + Computational geometry description of a set of spheres in Euclidean space. + + Each sphere can have a different radius. + + + + + + Integer which specifies the first index to be used for distinguishing + identifiers for spheres. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish spheres for explicit indexing. + + + + + + + + Geometric center of the spheres. This can be the center of mass. + Dimensionality and cardinality of the point and sphere set have to match. + The identifier_offset and identifier field of NXcg_point_set do not need + to be used as they should be same as the identifier_offset and the + identifier for the spheres. + + + + + + + + + In the case that all spheres have the same radius. + + + + + In the case that spheres have different radius use this + instead of the radius field. + + + + + + + + Reference to or definition of a coordinate system with + which the positions and directions are interpretable. + + + + + + Are the spheres closed or hollow? + + + + + + + + + + + + + + + + diff --git a/contributed_definitions/NXcg_tetrahedron_set.nxdl.xml b/contributed_definitions/NXcg_tetrahedron_set.nxdl.xml new file mode 100644 index 0000000000..b3e27b0f93 --- /dev/null +++ b/contributed_definitions/NXcg_tetrahedron_set.nxdl.xml @@ -0,0 +1,175 @@ + + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The cardinality of the set, i.e. the number of tetrahedra. + + + + + Computational geometry description of a set of tetrahedra in Euclidean space. + + The tetrahedra do not have to be connected. + As tetrahedral elements they are among hexahedral elements one of the most + frequently used geometric primitive for meshing and describing volumetric + and surface descriptions of objects at the continuum scale. + + A set of tetrahedra in 3D Euclidean space. + + The tetrahedra do not have to be connected, can have different size, + can intersect, and be rotated. + + Tetrahedra are the simplest and thus important geometrical primitive. + They are frequently used as elements in finite element meshing/modeling. + + Tetrahedra have to be non-degenerated, closed, and built of triangles + which are not self-intersecting. + + + + + + + + + + + Interior volume + + + + + + + + Position of the geometric center, which often is but not necessarily + has to be the center_of_mass of the tetrahedra. + + + + + + + + + Total surface area as the sum of all four triangular faces. + + + + + + + + Area of each of the four triangular faces of each tetrahedron. + + + + + + + + + Length of each edge of each tetrahedron. + + + + + + + + + Reference to or definition of a coordinate system with + which the qualifiers and mesh data are interpretable. + + + + + + Integer which specifies the first index to be used for distinguishing + tetrahedra. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish tetrahedra for explicit indexing. + + + + + + + + + + + + + A simple approach to describe the entire set of tetrahedra when the + main intention is to store the shape of the tetrahedra for visualization. + should take the possibility to describe + + + + + + Disentangled representations of the mesh of specific tetrahedra. + + + + + + Disentangled representation of the planar graph that each tetrahedron + represents. Such a description simplifies topological processing + or analyses of mesh primitive operations and neighborhood queries. + + + diff --git a/contributed_definitions/NXcg_triangle_set.nxdl.xml b/contributed_definitions/NXcg_triangle_set.nxdl.xml new file mode 100644 index 0000000000..3640f8ff30 --- /dev/null +++ b/contributed_definitions/NXcg_triangle_set.nxdl.xml @@ -0,0 +1,132 @@ + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + The dimensionality, which has to be at least 2. + + + + + The cardinality of the set, i.e. the number of triangles. + + + + + The number of unique vertices supporting the triangles. + + + + + Computational geometry description of a set of triangles in Euclidean space. + + + + + + + Reference to or definition of a coordinate system with + which the qualifiers and primitive data are interpretable. + + + + + Integer which specifies the first index to be used for distinguishing + triangles. Identifiers are defined either implicitly + or explicitly. For implicit indexing the identifiers are defined on the + interval [identifier_offset, identifier_offset+c-1]. + For explicit indexing the identifier array has to be defined. + + The identifier_offset field can for example be used to communicate if the + identifiers are expected to start from 1 (referred to as Fortran-/Matlab-) + or from 0 (referred to as C-, Python-style index notation) respectively. + + + + + Integer used to distinguish triangles for explicit indexing. + + + + + + + + + A simple approach to describe the entire set of triangles when the + main intention is to store the shape of the triangles for visualization. + + + + + + + + + + + + + + + Array of edge length values. For each triangle the edge length is + reported for the edges traversed according to the sequence + in which vertices are indexed in triangles. + + + + + + + + + Array of interior angle values. For each triangle the angle is + reported for the angle opposite to the edges which are traversed + according to the sequence in which vertices are indexed in triangles. + + + + + + + + + The center of mass of each polygon. + + + + + + + + + Axis-aligned or (approximate) (optimal) bounding boxes to each polygon. + + + diff --git a/contributed_definitions/NXcomponent_em.nxdl.xml b/contributed_definitions/NXcomponent_em.nxdl.xml deleted file mode 100644 index a68436e9b7..0000000000 --- a/contributed_definitions/NXcomponent_em.nxdl.xml +++ /dev/null @@ -1,69 +0,0 @@ - - - - - - - Base class for components used in an electron microscope. - - The electron microscope can be a real one or a simulated microscope. - The key motivation behind this generalization is the observation that in all - cases a controlled electron beam is generated in reality or that beam is simulated - and this beam is then used or modified in a controlled manner for the purpose - of studying physical interaction mechanisms of the beam with matter. - Here it does not matter whether one considers a real specimen or a simulated one. - - Using a common description for the real experiment in the lab and - what is - typically a simplification of it - via a computer simulation, has the benefit - that many pieces of information can be stored in the same way. In effect, - users are guided with finding information and unnecessary descriptive - variety for what are exactly the same concept is avoided to work towards - more interoperability. - - Another motivation to make no fundamental distinction between a scanning and - a transmission electron microscope is that both are electron microscopes whose - components are often very similar. - - - - Given name to the component e.g stage, lens C1, etc. - - - - - Ideally, a (globally) unique persistent identifier, link, or text to a - resource which gives further details to this component. - If such resource does not exist, a free-text field to report - further details about the component is possible. - - - - - - - Collection of axis-based translations and rotations to describe the - location and geometry of the component in the instrument. - - - diff --git a/contributed_definitions/NXcoordinate_system.nxdl.xml b/contributed_definitions/NXcoordinate_system.nxdl.xml new file mode 100644 index 0000000000..a8dcb83694 --- /dev/null +++ b/contributed_definitions/NXcoordinate_system.nxdl.xml @@ -0,0 +1,143 @@ + + + + + + Base class to detail a coordinate system (CS). + + Whenever possible, an instance of :ref:`NXcoordinate_system` should be used as + a member in an :ref:`NXcoordinate_system_set` and the name of the instance + should be this alias. This may support a process whereby jargon when talking + about coordinate systems and conventions may become cleaner for users + because it is not evident for people outside a lab that terms like e.g. + tip space or specimen space refer to the same coordinate system. + This is an example of jargon used in e.g. the field of atom + probe tomography. + + + + Human-readable field telling where the origin of this CS is. + Exemplar values could be *left corner of the lab bench*, *door-handle* + *pinhole through which the electron beam exists the pole piece*. + *barycenter of the triangle*, *center of mass of the stone*. + + + + + + An alternative name given to that coordinate system. + + + + + Coordinate system type. + + + + + + + + Handedness of the coordinate system if it is a Cartesian. + + + + + + + + + Possibility to define an alias for the name of the x-axis. + + + + + Human-readable field telling in which direction the x-axis points if that + instance of :ref:`NXcoordinate_system` has no reference to any parent and as such + is the mighty world reference frame. + + Exemplar values could be direction of gravity. + + + + + Base unit vector along the first axis which spans the coordinate system. + This axis is frequently referred to as the x-axis in real space and + the i-axis in reciprocal space. + + + + + + + + Possibility to define an alias for the name of the y-axis. + + + + + Human-readable field telling in which direction the y-axis points if that + instance of :ref:`NXcoordinate_system` has no reference to any parent and as such + is the mighty world reference frame. + + See docstring of x_alias for further details. + + + + + Base unit vector along the second axis which spans the coordinate system. + This axis is frequently referred to as the y-axis in real space and + the j-axis in reciprocal space. + + + + + + + + Possibility to define an alias for the name of the z-axis. + + + + + Human-readable field telling in which direction the z-axis points if that + instance of :ref:`NXcoordinate_system` has no reference to any parent and as such + is the mighty world reference frame. + + See docstring of x_alias for further details. + + + + + Base unit vector along the second axis which spans the coordinate system. + This axis is frequently referred to as the z-axis in real space and + the k-axis in reciprocal space. + + + + + + diff --git a/contributed_definitions/NXcrystal_structure.nxdl.xml b/contributed_definitions/NXcrystal_structure.nxdl.xml new file mode 100644 index 0000000000..4baccda138 --- /dev/null +++ b/contributed_definitions/NXcrystal_structure.nxdl.xml @@ -0,0 +1,280 @@ + + + + + + + + + Number of reflectors (Miller crystallographic plane triplets). + + + + + Number of atom positions. + + + + + Dimensionality of the lattice. + + + + + Base class to describe the atomic crystal structure of a phase. + + This base class contains key metadata that are relevant parameter to every + physics-based model to simulate radiation matter interaction. + + Examples where such base class is useful are kinematic or dynamic + diffraction simulations of e.g. (Kikuchi or other type of) patterns. + + + + Detail in which reference frame the unit cell is defined. + + + + + Dimensionality of the lattice. + + + + + + + + + + Reference to another resource that was used for + instantiating this structure model. + + + + + Crystallography unit cell parameters a, b, and c. + + + + + + + + + Crystallography unit cell parameters alpha, beta, and gamma. + + + + + + + + Area of the unit cell considering that d = 2. + + + + + Volume of the unit cell considering that d = 3. + + + + + Crystal system + + + + + + + + + + + + + + + Laue group using International Table of Crystallography Notation. + + + + + + Point group using International Table of Crystallography Notation. + + + + + + Space group from the International Table of Crystallography Notation. + + + + + + True if space group is considered a centrosymmetric one. + False if space group is considered a non-centrosymmetric one. + Centrosymmetric has all types and combinations of symmetry elements + (translation, rotational axis, mirror planes, center of inversion) + Non-centrosymmetric compared to centrosymmetric is constrained (no inversion). + Chiral compared to non-centrosymmetric is constrained (no mirror planes). + + + + + True if space group is considered a chiral one. + False if space group is consider a non-chiral one. + + + + + Identifier for each phase. + + The value 0 is reserved for the unknown phase that represents the + null-model no sufficiently significant confirmation. In other words, + the phase_name is n/a, notIndexed. + + The phase identifier value has to match with the integer postfix of the + group name which represents that instance in a NeXus/HDF5 file, i.e. + if two phases were used e.g. 0 and 1, two instances of an + :ref:`NXcrystal_structure` named phase0 and phase1 + should be stored in the HDF5 file. + + + + + + Name of the phase/alias. + + If the phase_identifier is 0 and one would like to use the field + phase_name the value should be n/a. + + + + + Label for each atom position. + + + + + + + + The hash value :math:`H` is :math:`H = Z + N*256` with :math:`Z` + the number of protons and :math:`N` the number of neutrons + of each isotope respectively. Z and N have to be 8-bit unsigned integers. + For the rationale behind this `M. Kühbach et al. (2021) <https://doi.org/10.1017/S1431927621012241>`_ + + + + + + + + + Atom positions. + + + + + + + + Reference to an instance of :ref:`NXcoordinate_system` + whereby the positions can be resolved. + + + + + + + Relative occupancy of the atom position. + + + + + + + + How many reflectors are distinguished. + + Value has to match value for symbol n_hkl. + + + + + + Miller indices :math:`(hkl)[uvw]` of the planes. + + The first triplet specify :math:`(hkl)` the second triplet :math:`[uvw]`. + Miller indices refer to the Cartesian right-handed coordinate system + of the unit cell. + + + + + + + + + Spacing between crystallographic planes as defined by field miller. + + + + + + + + Relative intensity of the signal for the plane. + + + + + + + + In case the :ref:`NXcrystal_structure` base class is used + with analyzed orientation maps this field stores how many scan points + of the map were identified as that phase. + + + + + + + diff --git a/contributed_definitions/NXcs_cpu_obj.nxdl.xml b/contributed_definitions/NXcs_cpu_obj.nxdl.xml new file mode 100644 index 0000000000..6ce5370a30 --- /dev/null +++ b/contributed_definitions/NXcs_cpu_obj.nxdl.xml @@ -0,0 +1,39 @@ + + + + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + + + Computer science description of a (central) processing unit (C)PU of a computer. + + + + Given name of the CPU. Users should be as specific as possible. + + + + diff --git a/contributed_definitions/NXcs_gpu_sys.nxdl.xml b/contributed_definitions/NXcs_cpu_sys.nxdl.xml similarity index 65% rename from contributed_definitions/NXcs_gpu_sys.nxdl.xml rename to contributed_definitions/NXcs_cpu_sys.nxdl.xml index e7fedabd41..0de162c661 100644 --- a/contributed_definitions/NXcs_gpu_sys.nxdl.xml +++ b/contributed_definitions/NXcs_cpu_sys.nxdl.xml @@ -2,9 +2,9 @@ - + The symbols used in the schema to specify e.g. dimensions of arrays. - Computer science description of a system of coprocessor or graphics processors. + Computer science description of a system of classical central processing units. + + For coprocessor or graphic cards use :ref:`NXcs_gpu_sys` instead. - + Granularizing at the socket level. - Typical examples follow: A desktop computer with a single GPU one - could describe using one instance of :ref:`NXcs_gpu_obj` inside one instance of - :ref:`NXcs_gpu_sys`. - A desktop computer with two GPUs one could describe using two instances - of :ref:`NXcs_gpu_obj` inside one instance of :ref:`NXcs_gpu_sys`. - A GPU server like nowadays used for artificial intelligence - one could describe as a system with n instances of :ref:`NXcs_gpu_obj` - in one :ref:`NXcs_gpu_sys` or :ref:`NXcs_cpu_sys`. + Typical examples follow: A desktop computer with a single CPU one + could describe using one instance of :ref:`NXcs_cpu_obj` inside one instance of + :ref:`NXcs_cpu_sys`. + A dual-socket server one could describe using two instances of :ref:`NXcs_cpu_obj` + inside one instance of :ref:`NXcs_cpu_sys`. + A server with two dual-socket server nodes one could describe + with the above group of one :ref:`NXcs_cpu_sys` into another :ref:`NXcs_cpu_sys`. diff --git a/base_classes/NXevent_data_em_set.nxdl.xml b/contributed_definitions/NXcs_gpu_obj.nxdl.xml similarity index 61% rename from base_classes/NXevent_data_em_set.nxdl.xml rename to contributed_definitions/NXcs_gpu_obj.nxdl.xml index e5f40c95a2..3c5b6c26ab 100644 --- a/base_classes/NXevent_data_em_set.nxdl.xml +++ b/contributed_definitions/NXcs_gpu_obj.nxdl.xml @@ -1,10 +1,10 @@ - + - + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + - Container to hold NXevent_data_em instances of an electron microscope session. - - An event is a time interval during which the microscope was configured, - considered stable, and used for characterization. + Computer science description of a graphic processing unit (GPU) of a computer. - + + + Given name of the GPU. Users should be as specific as possible. + + + diff --git a/contributed_definitions/NXcs_io_sys.nxdl.xml b/contributed_definitions/NXcs_io_sys.nxdl.xml index 59c620e7bc..5608c9f886 100644 --- a/contributed_definitions/NXcs_io_sys.nxdl.xml +++ b/contributed_definitions/NXcs_io_sys.nxdl.xml @@ -2,9 +2,9 @@ - + + + + The symbols used in the schema to specify e.g. dimensions of arrays. + + - Device to reduce an atmosphere to a controlled remaining pressure level. + Computer science description of a memory in a memory system. - - + + + Qualifier for the type of random access memory. + + + + - Principle type of the pump. + Total amount of data which the medium can hold. - - - - - - - + + + + Given name to the I/O unit. + + + diff --git a/contributed_definitions/NXcs_mm_sys.nxdl.xml b/contributed_definitions/NXcs_mm_sys.nxdl.xml index 6fed931cbe..d9c6779fd8 100644 --- a/contributed_definitions/NXcs_mm_sys.nxdl.xml +++ b/contributed_definitions/NXcs_mm_sys.nxdl.xml @@ -2,9 +2,9 @@ + + + + + Blue-print of a generic appdef for electron microscopy research formulated as a deep base class. + + This base class combines a method-specific and technical-design-level base class + instance to provide a template for storing parameterized descriptions of + pieces of information collected when performing electron microscopy research. + + The base class here shows all possible branches without making any statements + as to which of these have to be used in an instance. Thereby, the base class + provides a template how to name and structure concepts in a hierarchy + to support finding information and reducing the need for renaming and + restructuring information for a research field where many scientists perform + very specific research but who all also share commonalities like usage of + controlled electron beams, a focus on studies of electron beam matter interaction + to explore physical mechanisms and phenomena, or the desire to characterize materials + using electron microscopy. + + + + A collection of all programs and libraries which are considered relevant + to understand with which software tools this NeXus file instance was + generated. Ideally, to enable a binary recreation from the input data. + + Examples include the name and version of the libraries used to write the + instance. Ideally, the software that writes these :ref:`NXprogram` instances + also includes the version of the set of NeXus classes i.e. the specific + set of base classes, application definitions, and contributed definitions + with which the here described concepts can be resolved. + + For the `pynxtools library <https://github.com/FAIRmat-NFDI/pynxtools>`_ + which is used by the `NOMAD <https://nomad-lab.eu/nomad-lab>`_ + research data management system, it makes sense to store e.g. the GitHub + repository commit and respective submodule references used. + + + + + + The configuration of the I/O writer software (e.g. `pynxtools <https://github.com/FAIRmat-NFDI/pynxtools>`_) + which was used to generate this NeXus file instance. + + + + + + An at least as strong as SHA256 hashvalue of the file + which specifies the application definition. + + + + + NeXus NXDL schema to which this file conforms. + + + + + + + + Ideally, a (globally) unique persistent identifier + for referring to this experiment. + + An experiment should be understood in that this can be an experiment + in reality or a computer simulation because also the latter is an + experiment (see the Cambridge Dictionary experiment: + *a test done in order to find out something, eg if an idea is correct*). + + The identifier is usually issued by the facility, laboratory, + or the principle investigator. The identifier enables to link + experiments/simulations to e.g. proposals. + + + + + Free-text description about the experiment. + + Users are strongly advised to parameterize their description of the + experiment by using the respective base classes instead of writing prose + into this field. + + The reason is that such free-text field is difficult to machine-interpret. + The motivation behind keeping this field for now is to learn through + the information entered in this field in how far the current base + classes are incomplete. + + + + + ISO 8601 time code with local time zone offset to UTC information included + when the microscope session started. If the application demands that time + codes in this section of the application definition should only be used + for specifying when the experiment was performed - and the exact + duration is not relevant - this start_time field should be used. + + Often though it is useful to specify a time interval via setting both + a start_time and an end_time because this enables software tools and + users to collect a more detailed bookkeeping of the experiment. + + The user should be aware that even with having both time instances specified, + it may not be possible to infer how long the experiment took or for how + long data were acquired. + + More detailed timing data over the course of the experiment have + to be collected to compute this. These computations can take + advantage of individual time stamps start_time and end_time + in :ref:`NXevent_data_em` instances. + + + + + ISO 8601 time code with local time zone offset to UTC included when + the microscope session ended. See docstring of the start_time field + to see how the start_time and end_time should be used together. + + + + + + The program and eventual software libraries used with which the + NeXus instance was created. For the NOMAD OASIS research data + management system e.g. pynxtools and eventually all modules + if desired. + + + + + + Possibility to store a collection of data artifacts + associated with the experiment. + + + + + + Contact information and eventually details of at least one person + who performed or was involved in the session. This can be the + principle investigator who performed this experiment or the student + who performed the simulation. + Adding multiple users if relevant is recommended. + + + + Given (first) name and surname of the user. + + + + + Name of the affiliation of the user at the point in time + when the experiment was performed. + + + + + Postal address of the affiliation. + + + + + Email address of the user at the point in time when the experiment + was performed. Writing the most permanently used email is recommended. + + + + + Service as another mean of identification of a user than by the name. + Examples could be details about an ORCID or social media account of + the user. + + + + + (Business) (tele)phone number of the user at the point + in time when the experiment was performed. + + + + + Which role does the user have in the place and at the point + in time when the experiment was performed? Technician operating + the microscope, student, postdoc, principle investigator, or guest + are common examples. + + + + + + + A description of the material characterized in the experiment. + Sample and specimen are threaded as de facto synonyms. + Samples can be real specimens or virtual (see method). + + + + A qualifier whether the sample is a real one or a + virtual one (in a computer simulation) + + + + + + + + + + Ideally, (globally) unique persistent identifier which distinguishes + the specimen from all others and especially the predecessor/origin + from where the specimen was cut. + + This field must not be used for an alias! Instead, use name. + + In cases where multiple specimens were loaded into the microscope, + the identifier has to resolve the specific sample, whose results are + stored by this :ref:`NXentry` instance, because a single NXentry should be + used only for the characterization of a single specimen. + + Details about the specimen preparation should be + stored in resources referring to parent_identifier. + + + + + Identifier of the sample from which the sample was cut or the string + *None*. The purpose of this field is to support functionalities + for tracking sample provenance via a research data management system. + + + + + ISO 8601 time code with local time zone offset to UTC information + when the specimen was prepared. + + Ideally, report the end of the preparation, i.e. the last known time + the measured specimen surface was actively prepared. Ideally, this + matches the last timestamp that is mentioned in the digital resource + pointed to by parent_identifier. + + Knowing when the specimen was exposed to e.g. specific atmosphere is + especially required for environmentally sensitive material such as + hydrogen charged specimens or experiments including tracers with a + short half time. Additional time stamps prior to preparation_date + should better be placed in resources which describe but which do not pollute + the description here with prose. Resolving these connected pieces of information + is considered within the responsibility of the research data management + system. + + + + + An alias used to refer to the specimen to please readability for humans. + + + + + List of comma-separated elements from the periodic table that are + contained in the sample. If the sample substance has multiple + components, all elements from each component must be included in + `atom_types`. + + The purpose of the field is to offer research data management systems an + opportunity to parse the relevant elements without having to interpret + these from the resources pointed to by parent_identifier or walk through + eventually deeply nested groups in data instances. + + + + + + (Measured) sample thickness. + + The information is recorded to qualify if the beam used was likely + able to shine through the specimen. For scanning electron microscopy, + in many cases the specimen is typically thicker than what is + illuminatable by the electron beam. + + In this case the value should be set to the actual thickness of + the specimen viewed for an illumination situation where the nominal + surface normal of the specimen is parallel to the optical axis. + + + + + + + (Measured) density of the specimen. + + For multi-layered specimens this field should only be used to describe + the density of the excited volume. For scanning electron microscopy + the usage of this field is discouraged and instead an instance of an + :ref:`NXinteraction_vol_em` within individual :ref:`NXevent_data_em` + instances can provide a cleaner description of the relevant details + why one may wish to store the density of the specimen. + + + + + Discouraged free-text field to provide further detail although adding + parent_identifier and having a working research data management system + should provide this contextualization. + + + + + + + + + + + + A region-of-interest analyzed either during or after the session + for which specific processed data generated from the measured or the + simulated data are available. + + + + + + + + + + + + diff --git a/contributed_definitions/NXem_conventions_ebsd.nxdl.xml b/contributed_definitions/NXem_conventions_ebsd.nxdl.xml deleted file mode 100644 index d2d7456aaa..0000000000 --- a/contributed_definitions/NXem_conventions_ebsd.nxdl.xml +++ /dev/null @@ -1,230 +0,0 @@ - - - - - - - Base class for method-specific conventions EBSD. - - This base class is expected to be used with :ref:`NXem_conventions`. - - This is the main issue which currently is not in all cases documented - and thus limits the interoperability and value of collected EBSD data. - Not communicating EBSD data with such contextual pieces of information - and the use of file formats which do not store this information is the - key unsolved problem. - - - - Details about the gnomonic projection reference frame. - - - - Type of coordinate system/reference frame used for identifying - positions in the gnomonic projection space Xg, Yg, Zg - according to DOI: 10.1016/j.matchar.2016.04.008. - - - - - - - - - Handedness of coordinate system. - - - - - - - - - Is the origin of the gnomonic coordinate system located - where we assume the location of the pattern centre. - This is the location Xg = 0, Yg = 0, Zg = 0 according to - reference DOI: 10.1016/j.matchar.2016.04.008. - - - - - - - - - Direction of the positively pointing "gnomomic" x-axis base - vector when viewing how the diffraction pattern looks on the - detector screen. We assume the configuration is inspected by - looking towards the sample surface from a position - that is located behind the detector. - Different tools assume that different strategies can be used - and are perceived as differently convenient to enter - details about coordinate system definitions. In this ELN users - have to possibility to fill in what they assume is sufficient to - define the coordinate system directions unambiguously. - Software which works with this user input needs to offer parsing - capabilities which detect conflicting input and warn accordingly. - - - - - - - - - - - - - - Direction of the positively pointing "gnomomic" y-axis base - vector when viewing how the diffraction pattern looks on the - detector screen. We assume the configuration is inspected by - looking towards the sample surface from a position - that is located behind the detector. - For further information consult also the help info for the - xaxis_direction field. - - - - - - - - - - - - - - Direction of the positively pointing "gnomomic" z-axis base - vector when viewing how the diffraction pattern looks on the - detector screen. We assume the configuration is inspected by - looking towards the sample surface from a position - that is located behind the detector. - For further information consult also the help info for the - xaxis_direction field. - - - - - - - - - - - - - - - Details about the definition of the pattern centre - as a special point in the gnomonic projection reference frame. - - - - From which border of the EBSP (in the detector reference frame) - is the pattern centre's x-position (PCx) measured? - Keywords assume the region-of-interest is defined by - a rectangle. We observe this rectangle and inspect the - direction of the outer-unit normals to the edges of - this rectangle. - - - - - - - - - - - - In which direction are positive values for PCx measured from - the specified boundary. Keep in mind that the gnomonic space - is in virtually all cases embedded in the detector space. - Specifically, the XgYg plane is defined such that it is - embedded/laying inside the XdYd plane (of the detector - reference frame). - When the normalization direction is the same as e.g. the - detector x-axis direction, we state that we effectively - normalize in fractions of the width of the detector. - - The issue with terms like width and height is that these - degenerate if the detector region-of-interest is square-shaped. - This is why we should better avoid talking about width and height but - state how we would measure distances practically with a ruler and - how we then measure positive distances. - - - - - - - - - - - - From which border of the EBSP (in the detector reference - frame) is the pattern centre's y-position (PCy) measured? - For further details inspect the help button of - xaxis_boundary_convention. - - - - - - - - - - - - In which direction are positive values for PCy measured from - the specified boundary. - For further details inspect the help button of - xaxis_normalization_direction. - - - - - - - - - - - - diff --git a/contributed_definitions/NXem_correlation.nxdl.xml b/contributed_definitions/NXem_correlation.nxdl.xml new file mode 100644 index 0000000000..58c0f68255 --- /dev/null +++ b/contributed_definitions/NXem_correlation.nxdl.xml @@ -0,0 +1,245 @@ + + + + + + Base class to combine different method-specific data in electron microscopy. + + This base class represent a template for documenting correlations + (spatial, temporal) between different method-specific results. + + + + Details about processing steps. + + + + + + Details about correlated or logically connected EBSD datasets. + + One important class of such correlated experiments are the so-called + (quasi) in-situ experiments. In this case the same or nearly the same ROI + gets analyzed via a repetitive sequence of thermomechanical treatment, + sample preparation, measurement, on-the-fly-indexing. Phenomena + investigated are recrystallization, strain accumulation, material damage. + Post-processing is required to correlate and reidentify eventual + microstructural features or local ROIs across several orientation maps. + + Another important class of correlated experiments are the so-called + serial-sectioning experiments. Here the same sample is measured + repetitively after polishing each time, to create a stack of + orientation data which can be reconstructed to a + three-dimensional volume ROI. + + Data can be correlated in time, position (spatial), or both (spatiotemporal). + + Spatial correlations between repetitively characterized regions-of-interests + are typically correlated using image registration and alignment algorithms. + For this typically so-called landmarks are used. These can be grains with + a very large size or specific shape, i.e. grains which are qualitatively + different enough to be used as a guide how images are shifted relative to + one another. Other commonly used landmarks are fiducial marks which are + milled into the specimen surface using focus-ion beam milling and/or various + types of indentation methods. + + As far as the same physical region-of-interest is just measured several times, + the additional issue of the depth increment is not a concern. However, correct + assumptions for the depth increment, amount of material removed along the milling + direction is relevant for accurate and precise three-dimensional (serial-sectioning) + correlations. For these studies it can be tricky though to assume or estimate + useful depth increments. Different strategies have been proposed like + calibrations, wedged-shaped landmarks and computer simulation assisted + assumption making. + + Despite the use of landmarks, there are many practical issues which make the + processing of correlations imprecise and inaccurate. Among these are drift + and shift of the specimen, instabilities of the holder, the beam, irrespective + of the source of the drift, charging effects, here specifically causing local + image distortions and rotations which may require special processing algorithms + to reduce such imprecisions. + + Time correlations face all of the above-mentioned issues surplus the challenge + that specific experimental protocols have to be used to ensure the material state + is observed at specific physical time. The example of quasi in-situ characterization + of crystal growth phenomena, a common topic in engineering or modern catalysis research + makes it necessary to consider that e.g. the target value for the desired annealing + temperature is not just gauged based on macroscopic arguments but considers + that transient effects take place. Heating or quenching a sample might thus might + not have been executed under conditions in the interaction volume as they are + documented and/or assumed. + + These issue cause that correlations have an error margin as to how accurately + respective datasets were not only just synced based on the geometry of the + region-of-interests and the time markers but also to asssure which physical + conditions the specimen experienced over the course of the measurements. + + The fourth example of the em_om reference implementation explores the use of the + correlation group with a serial-sectioning datasets that was collected by the + classical Inconel 100 dataset collected by M. D. Uchic and colleagues + (M. Groeber M, Haley BK, Uchic MD, Dimiduk DM, Ghosh S 3d reconstruction and + characterization of polycrystalline microstructures using a fib-sem system data set. + Mater Charac 2006, 57 259–273. 10.1016/j.matchar.2006.01.019M). + + This dataset was specifically relevant in driving forward the implementation + of the DREAM.3D software. DREAM.3D is an open-source software project for + post-processing and reconstructing, i.e. correlating sets of orientation + microscopy data foremost spatially. One focus of the software is the + (post-)processing of EBSD datasets. Another cutting edge tool with similar + scope but a commercial solution by Bruker is QUBE which was developed by + P. Konijnenberg and coworkers. + + Conceptually, software like DREAM.3D supports users with creating linear + workflows of post-processing tasks. Workflows can be instructed via the + graphical user interface or via so-called pipeline processing via command line + calls. DREAM.3D is especially useful because its internal system documents all + input, output, and parameter of the processing steps. This makes DREAM.3D a + good candidate to interface with tools like em_om parser. Specifically, DREAM.3D + documents numerical results via a customized HDF5 file format called DREAM3D. + Workflow steps and settings are stored as nested dictionaries in JSON syntax + inside a supplementary JSON file or alongside the data in the DREAM3D file. + DREAM.3D has a few hundred algorithms implemented. These are called filters + in DREAM.3D terminology. + + Users configure a workflow which instructs DREAM.3D to send the data through + a chain of predefined and configured filters. Given that for each analysis + the filter is documented via its version tags surplus its parameter and setting + via a controlled vocabulary, interpreting the content of a DREAM3D HDF5 file + is possible in an automated manner using a parser. This makes DREAM.3D analyses + repeatable and self-descriptive. A key limitation though is that most frequently + the initial set of input data come from commercial files like ANG. + This missing link between the provenance of these input files, their associated + creation as electron microscope session, is also what NXem_ebsd solves. + + Nevertheless, as this can be solved with e.g. NXem_ebsd we are convinced that + the DREAM.3D and the em_om parser can work productively together to realize + RDMS-agnostic parsing of serial-section analyses. + + The internal documentation of the DREAM.3D workflow also simplifies the + provenance tracking represented by an instance of NXem_ebsd as not every + intermediate results has to be stored. Therefore, the fourth example + focuses on the key result obtained from DREAM.3D - the reconstructed + and aligned three-dimensional orientation map. + + Usually, this result is the starting point for further post-processing + and characterization of structural features. As here orientation microscopy + is insofar scale invariant using DREAM.3D, NXem_ebsd, and em_om should + be useful for different characterization methods, such as EBSD, Transmission + Kikuchi Diffraction (TKD), Automated Crystal Orientation Mapping (ACOM), + Nanobeam Electron Diffraction (using commercial systems like NanoMegas ASTAR) + or open-source implementations of these techniques (such as via pyxem/orix). + + The result of orientation microscopy methods are maps of local orientation + and thermodynamic phase (crystal structure) pieces of information. Virtually + all post-processing of such results for structural features includes again + a workflow of steps which are covered though by the NXms partner application + definition. The respective source of the data in an instance of NXms can + again be a link or reference to an instance of NXem_ebsd to complete the + chain of provenance. + + + + + + Descriptor representing the image contrast. + + + + + + Title of the default plot. + + + + + Descriptor values displaying the ROI. + + + + + + + + + + Descriptor values. + + + + + + Calibrated coordinate along the z-axis. + + + + + + + Label for the z axis + + + + + + Calibrated coordinate along the y-axis. + + + + + + + Label for the y axis + + + + + + Calibrated coordinate along the x-axis. + + + + + + + Label for the x axis + + + + + + + diff --git a/contributed_definitions/NXem_eds.nxdl.xml b/contributed_definitions/NXem_eds.nxdl.xml new file mode 100644 index 0000000000..735bfe8979 --- /dev/null +++ b/contributed_definitions/NXem_eds.nxdl.xml @@ -0,0 +1,144 @@ + + + + + + + + + + Number of pixel along the y direction, the slow direction + + + + + Number of pixel along the x direction, the fast direction + + + + + Number of X-ray photon energy (bins), the fastest direction. + + + + + Number of identified elements + + + + + Number of peaks detected + + + + + Base class method-specific for energy-dispersive X-ray spectroscopy (EDS/EDX). + + `IUPAC instead of Siegbahn notation <https://doi.org/10.1002/xrs.1300200308>`_ should be used. + + + + + Details about computational steps how peaks were indexed as elements. + + + + The program with which the indexing was performed. + + + + + Name and location of each X-ray line which was indexed as a known ion. + For each ion, an NXion instance should be created which specifies + the origin of the signal. For each ion also the relevant IUPAC notation + X-ray lines should be specified. + + + + + Associated lower :math:`[e_{min}, e_{max}]` bounds of the + energy which is assumed associated with this peak. + + + + + + + + Theoretical energy of the line according to IUPAC. + + + + + IUPAC notation identifier of the line which the peak represents. + + This can be a list of IUPAC notations for (the seldom) case that + multiple lines are grouped with the same peak. + + + + + + + + + + List of the names of identified elements. + + + + + + + + Individual element-specific EDS/EDX/EDXS/SXES mapping + + A composition map is an image whose intensities for each pixel are the + accumulated X-ray quanta *under the curve(s)* of a set of peaks. + + These element-specific EDS maps are NXimage_r_set instances + and need to be named with the name of the element from the + element_names field. + + + + + A list of NXpeak instance names whose X-ray quanta + where accumulated for each pixel which yields an element-specific + EDS map. + + + + + + + + + + diff --git a/contributed_definitions/NXem_eels.nxdl.xml b/contributed_definitions/NXem_eels.nxdl.xml new file mode 100644 index 0000000000..c355f6fd62 --- /dev/null +++ b/contributed_definitions/NXem_eels.nxdl.xml @@ -0,0 +1,79 @@ + + + + + + + + Number of electron energy loss bins. + + + + + Base class method-specific for Electron Energy Loss Spectroscopy (EELS). + + + + + NXspectrum_set_em specialized for EELS. + + + + + + + + + + Energy loss. + + + + + + + + + Energy loss. + + + + + + + Energy loss. + + + + + + + diff --git a/contributed_definitions/NXem_adf.nxdl.xml b/contributed_definitions/NXem_img.nxdl.xml similarity index 75% rename from contributed_definitions/NXem_adf.nxdl.xml rename to contributed_definitions/NXem_img.nxdl.xml index 1106424132..ebf17380a6 100644 --- a/contributed_definitions/NXem_adf.nxdl.xml +++ b/contributed_definitions/NXem_img.nxdl.xml @@ -2,9 +2,9 @@ - + @@ -40,7 +40,7 @@ - Base class method-specific for annular dark field imaging. + Base class for method-specific generic imaging. In the majority of cases simple d-dimensional regular scan patterns are used to probe a region-of-interest (ROI). Examples can be single point aka spot @@ -50,16 +50,14 @@ For now the base class provides for scans for which the settings, binning, and energy resolution is the same for each scan point. - - - - Annulus inner (first value) and outer (second value) half angle. - - - - - - - + + + Which imaging mode was used? + + + + + + + diff --git a/contributed_definitions/NXem_method.nxdl.xml b/contributed_definitions/NXem_method.nxdl.xml new file mode 100644 index 0000000000..086d4833d2 --- /dev/null +++ b/contributed_definitions/NXem_method.nxdl.xml @@ -0,0 +1,47 @@ + + + + + + + Base class to describe specific analysis methods in electron microscopy. + + This base class represent a template how specialized, deep, and method-specific + base classes can be defined with which an (NXem) application + definition gets equipped with specific groups to document method-specific + processing steps and report analyzed quantities. + + The template base class name :ref:`NXem_method` needs to be changed for each + method e.g. :ref:`NXem_adf`, :ref:`NXem_ebsd`, :ref:`NXem_eels`, :ref:`NXem_eds`. + + + + Details about processing steps. + + + + + + + diff --git a/contributed_definitions/NXem_msr.nxdl.xml b/contributed_definitions/NXem_msr.nxdl.xml new file mode 100644 index 0000000000..a6442b1e2e --- /dev/null +++ b/contributed_definitions/NXem_msr.nxdl.xml @@ -0,0 +1,96 @@ + + + + + + Base class for collecting a session with a real electron microscope. + + For collecting data and experiments which are simulations of an + electron microscope use the :ref:`NXem_sim` base class. + + + + (Meta)data of the microscope and the lab in which it stands. + + This em_lab group differs from potential em_lab groups inside + :ref:`NXevent_data_em` instances in that here the more static descriptions + are kept while changing, i.e. time-dependent pieces of information are + logged, via the em_lab group inside the desired number of instances + of NXevent_data_em. + + While using an :ref:`NXevent_data_em` instance, users should store only those + settings about a component which are relevant to understand the current + state of the component. Here, current means for the time interval which + the event covers (as it is detailed via start_time and end_time) timestamps. + + For example it is not relevant to store in each :ref:`NXevent_data_em` + electron_source group again the details of the gun type and the manufacturer + but only the high-voltage value and that only if it is different from the value + that is specified in the em_lab section for the static settings. + + In effect, this defines an information inference hierarchy which starts + in an individual :ref:`NXevent_data_em` instance followed by a probing of the + static section. + + + + Given name of the microscope at the hosting institution. + This is an alias. Examples could be NionHermes, Titan, JEOL, + Gemini, etc. + + + + + Location of the lab or place where the instrument is installed. + Using GEOREF is preferred. + + + + + + + + + + Description of the type of the detector. + + Electron microscopes have typically multiple detectors. + Different technologies are in use like CCD, scintillator, + direct electron, CMOS, or image plate to name but a few. + + + + Instrument-specific alias/name + + + + + + + + + + diff --git a/contributed_definitions/NXem_sim.nxdl.xml b/contributed_definitions/NXem_sim.nxdl.xml new file mode 100644 index 0000000000..f5f10b1b95 --- /dev/null +++ b/contributed_definitions/NXem_sim.nxdl.xml @@ -0,0 +1,60 @@ + + + + + + + Base class for simulating electron microscopy relevant beam-matter interaction. + + The concept behind this base class is to keep it as generic as possible + that simulations of electron/ion beam interaction with matter can be + represented. This base class is envisioned as the twin of the :ref:`NXem_msr` + base class. + + It is an attempt to test the idea if at some point one might even use the + same base class template to describe measurements and computer simulations + of electron microscopy. This idea is attractive because the only practical + difference between a description of a measurement with a microscope and a + computer simulation is that the latter is typically a substantially simplified + representation of the real microscope surplus the focus of the research + in such cases on specific questions. + + Such simplification can be with respect to the optical setup, typically the + ignoring of the fact that the electron beam is produced by a complex setup + of lenses while in simulations often single Einzel lenses are considered. + Dynamics of the environment like temperature fluctuation in a lab, vibrations, + users, and multiple detectors are typically either ignored or reduced in + complexity and number and coming with idealizations to keep the simulations + focused on the specific reason questions and efficiently numerically executable. + + + + Details about the simulation. + + + + + + + diff --git a/contributed_definitions/NXgraph_node_set.nxdl.xml b/contributed_definitions/NXgraph_node_set.nxdl.xml index fb04291647..9b98765d21 100644 --- a/contributed_definitions/NXgraph_node_set.nxdl.xml +++ b/contributed_definitions/NXgraph_node_set.nxdl.xml @@ -2,9 +2,9 @@ + + + + + Number of images in the (hyper)stack. + + + + + Number of pixel per image in the slowest direction. + + + + + Number of pixel per image in the slow direction. + + + + + Number of pixel per image in the fast direction. + + + + + Specialized base class container for reporting a set of images in reciprocal space. + + In practice, complex numbers are encoded via some formatted pair of real values. + Typically, fast Algorithms for computing Fourier Transformations (FFT) are + used to encode images in reciprocal (frequency) space. FFT libraries are used + for implementing the key functionalities of these mathematical operations. + + Different libraries use different representations and encoding of the + image computed. Details can be found in the respective sections of the + typical FFT libraries: + + * `FFTW by M. Frigo and S. G. Johnson <https://www.fftw.org/fftw3_doc/Tutorial.html#Tutorial>`_ + * `Intel MKL by the Intel Co. <https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-c/2023-0/fourier-transform-functions.html>`_ + * `cuFFT by the NVidia Co. <https://docs.nvidia.com/cuda/cufft/index.html>`_ + + Users are strongly advised to inspect carefully which specific conventions + their library uses to be able to store and modify the implementation of their + code so that the serialized representations as it is detailed + here for NeXus matches with their intention. + + One- and two-dimensional FFTs should use the stack(NXdata) instances. + Three-dimensional FFTs should use the hyperstack(NXdata) instances. + + + + + Image stack. + + + + Image intensity of the real part. + + + + + + + + + + Image intensity of the imaginary part. + + + + + + + + + + Magnitude of the image intensity. + + + + + + + + + + Image identifier + + + + + + + Image identifier. + + + + + + + Pixel coordinate center along j direction. + + + + + + + Coordinate along j direction. + + + + + + Pixel coordinate center along i direction. + + + + + + + Coordinate along i direction. + + + + + + + Image hyperstack. + + + + Image intensity of the real part. + + + + + + + + + + + Image intensity of the imaginary part. + + + + + + + + + + + Magnitude of the image intensity. + + + + + + + + + + + Image identifier + + + + + + + Image identifier. + + + + + + Pixel coordinate center along k direction. + + + + + + + Coordinate along j direction. + + + + + + Pixel coordinate center along j direction. + + + + + + + Coordinate along j direction. + + + + + + Pixel coordinate center along i direction. + + + + + + + Coordinate along i direction. + + + + + diff --git a/base_classes/NXimage_set.nxdl.xml b/contributed_definitions/NXimage_r_set.nxdl.xml similarity index 61% rename from base_classes/NXimage_set.nxdl.xml rename to contributed_definitions/NXimage_r_set.nxdl.xml index a482480c85..3ef371809d 100644 --- a/base_classes/NXimage_set.nxdl.xml +++ b/contributed_definitions/NXimage_r_set.nxdl.xml @@ -1,9 +1,9 @@ - + - + @@ -40,42 +40,14 @@ - Container for reporting a set of images taken. + Specialized base class container for reporting a set of images in real space. - - - Details how images were processed from the detector readings. - - - - Resolvable data artifact (e.g. filename) from which the all values in - the NXdata instances in this NXimage_set were loaded during parsing. - - - - An at least as strong as SHA256 hashvalue of the data artifact which - source represents digitally to support provenance tracking. - - - - - - Imaging (data collection) mode of the instrument during acquisition - of the data in this NXimage_set instance. - - - - - Link or name of an NXdetector instance with which the data were collected. - - - - + Image (stack). - + Image intensity values. @@ -85,14 +57,14 @@ - + Image identifier - + Image identifier. @@ -100,12 +72,12 @@ - Pixel coordinate center of mass along y direction. + Pixel coordinate center along y direction. - + Coordinate along y direction. @@ -113,12 +85,12 @@ - Pixel coordinate center of mass along x direction. + Pixel coordinate center along x direction. - + Coordinate along x direction. diff --git a/contributed_definitions/NXimage_r_set_diff.nxdl.xml b/contributed_definitions/NXimage_r_set_diff.nxdl.xml deleted file mode 100644 index 98919c2ed9..0000000000 --- a/contributed_definitions/NXimage_r_set_diff.nxdl.xml +++ /dev/null @@ -1,179 +0,0 @@ - - - - - - - - Number of scanned points. Scan point may have none, one, or more pattern. - - - - - Number of diffraction pattern. - - - - - Number of pixel per pattern in the slow direction. - - - - - Number of pixel per pattern in the fast direction. - - - - - Base class specialized for reporting a cuboidal stack of diffraction pattern. - - Diffraction pattern, whether they were simulated or measured are the raw data - for computational workflows to characterize the phase and orientation - of crystalline regions in matter. - - Steps of post-processing the diffraction patterns should be documented using - method-specific specialized base classes. All eventual post-processing of - resulting orientation maps (2D or 3D) should be documented via :ref:`NXms_recon`. - - To implement an example how this base class can be used we focused in FAIRmat - on Kikuchi diffraction pattern here specifically the research community - of Electron Backscatter Diffraction (EBSD). - - For this reason, this base class and related :ref:`NXem_base` classes extend the - work of `M. A. Jackson et al. <https://doi.org/10.1186/2193-9772-3-4>`_ - (one of the developers of DREAM.3D) and the H5OINA public file format developed by - `P. Pinard et al. <https://doi.org/10.1017/S1431927621006103>`_ with Oxford Instruments. - - Kikuchi pattern are typically collected with FIB/SEM microscopes, - for two- and three-dimensional orientation microscopy. - - For a detailed overview of these techniques see e.g. - - * `M. A. Groeber et al. <https://doi.org/10.1186/2193-9772-3-5>`_ - * `A. J. Schwartz et al. <https://doi.org/10.1007/978-1-4757-3205-4>`_ - * `P. A. Rottman et al. <https://doi.org/10.1016/j.mattod.2021.05.003>`_ - - Serial-sectioning demands a recurrent sequence of ion milling and measuring. - This suggests that each serial section is at least an own NXevent_data_em - instance. The three-dimensional characterization as such demands a computational - step where the maps for each serial section get cleaned, aligned, and registered - into an image stack. This image stack represents a digital model of the - inspected microstructural volume. Often this volume is called a (representative) - volume element (RVE). Several software packages exists for performing - these post-processing tasks. - - This example may inspire users of other types of diffraction methods. - - - - Category which type of diffraction pattern is reported. - - - - - - - - Collected diffraction pattern as an image stack. As raw and closest to the - first retrievable measured data as possible, i.e. do not use this - container to store already averaged, filtered or whatever post-processed - pattern unless these are generated unmodifiably in such manner by the - instrument given the way how the instrument and control software - was configured for your microscope session. - - - - Array which resolves the scan point to which each pattern belongs. - - Scan points are evaluated in sequence starting from scan point zero - until scan point n_sc - 1. Evaluating the cumulated of this array - decodes which pattern in intensity belongs to which scan point. - - Take an example with three scan points: The first scan point has one - pattern, the second has three pattern, the last scan point has no - pattern. In this case the scan_point_identifier are 0, 1, 1, 1. - The length of the scan_point_identifier array is four because four - pattern were measured in total. - - In most cases usually one pattern is averaged by the detector for - some amount of time and then reported as one pattern. - - - - - - - - Intensity of the diffraction pattern. - - - - - - - - - - Pattern intensity - - - - - - - Pattern are enumerated starting from 0 to n_p - 1. - - - - - - - Pattern identifier - - - - - - - diff --git a/contributed_definitions/NXinteraction_vol_em.nxdl.xml b/contributed_definitions/NXinteraction_vol_em.nxdl.xml new file mode 100644 index 0000000000..a6beeb6482 --- /dev/null +++ b/contributed_definitions/NXinteraction_vol_em.nxdl.xml @@ -0,0 +1,37 @@ + + + + + Base class for storing details about a modelled shape of interaction volume. + + The interaction volume is mainly relevant in scanning electron microscopy + when the sample is thick enough so that the beam is unable to illuminate + through the specimen. + Computer models like Monte Carlo or molecular dynamics / electron beam + interaction simulations can be used to qualify and/or quantify the shape of + the interaction volume. + + Explicit or implicit descriptions are possible. + + * An implicit description is via a set of electron/specimen interactions + represented ideally as trajectory data from the computer simulation. + * An explicit description is via an iso-contour surface using either + a simulation grid or a triangulated surface mesh of the approximated + iso-contour surface evaluated at specific threshold values. + Iso-contours could be computed from electron or particle fluxes through + an imaginary control surface (the iso-surface). + Threshold values can be defined by particles passing through a unit control + volume (electrons) or energy-levels (e.g. the case of X-rays). + Details depend on the model. + * Another explicit description is via theoretical models which may + be relevant e.g. for X-ray spectroscopy + + Further details on how the interaction volume can be quantified + is available in the literature for example: + + * `S. Richter et al. <https://doi.org/10.1088/1757-899X/109/1/012014>`_ + * `J. Bünger et al. <https://doi.org/10.1017/S1431927622000083>`_ + + + + diff --git a/contributed_definitions/NXisocontour.nxdl.xml b/contributed_definitions/NXisocontour.nxdl.xml index d42b1ab9e3..8c4361d279 100644 --- a/contributed_definitions/NXisocontour.nxdl.xml +++ b/contributed_definitions/NXisocontour.nxdl.xml @@ -2,9 +2,9 @@ +interpretation (fundamentally this is an assumption) and can differ for different descriptors--> The simulated or characterized material volume element aka domain. At least one instance of geometry required either NXcg_grid, diff --git a/contributed_definitions/NXms_ipf.nxdl.xml b/contributed_definitions/NXms_ipf.nxdl.xml deleted file mode 100644 index 0be3dc81e4..0000000000 --- a/contributed_definitions/NXms_ipf.nxdl.xml +++ /dev/null @@ -1,383 +0,0 @@ - - - - - - - - - Number of pixel along the z slowest direction. - - - - - Number of pixel along the y slow direction. - - - - - Number of pixel along the x fast direction. - - - - - Number of RGB values along the fastest direction, always three. - - - - - Dimensionality of the mapping (either 2 or 3). - - - - - Base class to store an inverse pole figure (IPF) mapping (IPF map). - - - - Reference to the coordinate system whereby the projection_direction is defined. - - If the field depends_on is not provided but a parent of the instance - of this base class or its specialization defines an :ref:`NXcoordinate_system_set` - and exactly one :ref:`NXcoordinate_system`, the reference points to this system. - - If nothing is provided and none of the above-mentioned references pointing - in a parent, McStas is assumed. - - - - - The direction along which orientations are projected. - - - - - - - - Details about the original grid. - - Here original grid means the one onto which the IPF map was computed - when exported from the tech partner's file format representation. - - - - - Details about the grid onto which the IPF is recomputed. - - Rescaling the visualization of the IPF map may be needed to enable - visualization in specific software tools like H5Web. - The value specifies the fractional change of the spacing between - the original mapping and the scaled one. - - - - - How where orientation values at the location of the output grid - positions computed. - - Nearest neighbour means the orientation of the closed (Euclidean distance) - grid point of the input_grid was taken. - - - - - - - - Inverse pole figure mapping. - - Default inverse pole figure (IPF) plot of the data specific for each - phase. No ipf_mapID instances for non-indexed scan points as these are - by definition assigned the null phase with phase_identifier 0. - Inspect the definition of :ref:`NXcrystal_structure` and its field - phase_identifier for further details. - - Details about possible regridding and associated interpolation - during the computation of the IPF map visualization can be stored - using the input_grid, output_grid, and interpolation fields. - - The main purpose of this map is to offer a normalized default representation - of the IPF map for consumption by a research data management system (RDMS). - This is aligned with the first aim of :ref:`NXms_ipf`, to bring colleagues and - users of IPF maps together to discuss which pieces of information - need to be stored together. We are convinced a step-by-step design and - community-driven discussion about which pieces of information should - and/or need to be included is a practical strategy to work towards an - interoperable description and data model for exchanging IPF maps as specific - community-accepted tools to convey orientation maps. - - With this design the individual RDMS solutions and tools can still continue - to support specific custom data analyses workflow and routes but at least - there is one common understanding which enables also those users who are - not necessarily experts in all the details of the underlying techniques - can understand and thus eventually judge if the dataset is worth to be - reused or repurposed. - - - - - - Inverse pole figure color code for each map coordinate. - - - - - - - - - - Pixel center coordinate calibrated for step size along the z axis of the map. - - - - - - - - - Pixel center coordinate calibrated for step size along the y axis of the map. - - - - - - - - - Pixel center coordinate calibrated for step size along the x axis of the map. - - - - - - - - - - The color code which maps colors into orientation into the fundamental zone. - - For each stereographic standard triangle (SST), i.e. a rendering of the - fundamental zone of the crystal-symmetry-reduced orientation space - SO3, it is possible to define a color model which assigns each point in - the fundamental zone a color. - - Different mapping models are used. These implement (slightly) different - scaling relations. Differences exist across representations of tech partners. - - Differences are which base colors of the RGB color model are placed in - which extremal position of the SST and where the white point is located. - - For further details see: - - * [G. Nolze et al.](https://doi.org/10.1107/S1600576716012942) - * Srikanth Patala and coworkers"'" work and of others. - - Details are implementation-specific and not standardized yet. - - Given that the SST has a complicated geometry, it cannot yet be - visualized using tools like H5Web, which is why for now the matrix - of a rasterized image which is rendered by the backend tool gets - copied into an RGB matrix to offer a default plot. - - - - - - Inverse pole figure color code for each map coordinate. - - - - - - - - - - Pixel along the y-axis. - - - - - - - - - Pixel along the x-axis. - - - - - - - - - diff --git a/contributed_definitions/NXms_odf_set.nxdl.xml b/contributed_definitions/NXms_ipf_set.nxdl.xml similarity index 80% rename from contributed_definitions/NXms_odf_set.nxdl.xml rename to contributed_definitions/NXms_ipf_set.nxdl.xml index c21aa36985..776eb0a6c9 100644 --- a/contributed_definitions/NXms_odf_set.nxdl.xml +++ b/contributed_definitions/NXms_ipf_set.nxdl.xml @@ -2,9 +2,9 @@ - + - Base class to group multiple :ref:`NXms_odf` instances. + Base class to group multiple :ref:`NXms_ipf` instances. - A collection of orientation distribution function approximations. + A collection of inverse pole figure approximations. - + diff --git a/contributed_definitions/NXms_mtex_config.nxdl.xml b/contributed_definitions/NXms_mtex_config.nxdl.xml deleted file mode 100644 index 90e4132bee..0000000000 --- a/contributed_definitions/NXms_mtex_config.nxdl.xml +++ /dev/null @@ -1,310 +0,0 @@ - - - - - - Base class to store the configuration when using the MTex/Matlab software. - - MTex is a Matlab package for texture analysis used in the Materials and Earth Sciences. - See `R. Hielscher et al. <https://mtex-toolbox.github.io/publications>`_ and - the `MTex source code <https://github.com/mtex-toolbox>`_ for details. - - - - Reference frame and orientation conventions. - Consult the `MTex docs <https://mtex-toolbox.github.io/EBSDReferenceFrame.html>`_ for details. - - - - TODO with MTex developers - - - - - - TODO with MTex developers - - - - - - TODO with MTex developers - - - - - - TODO with MTex developers - - - - - - TODO with MTex developers - - - - - - - - - - - Settings relevant for generating plots. - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - True if MTex renders a scale bar with figures. - - - - - True if MTex renders a grid with figures. - - - - - Code for the function handle used for annotating pole figure plots. - - - - - TODO with MTex developers - - - - - - - - - TODO with MTex developers - - - - - - - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - - Miscellaneous other settings of MTex. - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - - - Miscellaneous settings relevant for numerics. - - - - Return value of the Matlab eps command. - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - - Miscellaneous settings relevant of the system where MTex runs. - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - TODO with MTex developers - - - - - - Collection of paths from where MTex reads information and code. - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - Absolute path to specific component of MTex source code. - - - - - List of file type suffixes for which MTex assumes - texture/pole figure information. - - - - - List of file type suffixes for which MTex assumes EBSD content. - - - - - diff --git a/contributed_definitions/NXms_odf.nxdl.xml b/contributed_definitions/NXms_odf.nxdl.xml deleted file mode 100644 index 3e9d11dd70..0000000000 --- a/contributed_definitions/NXms_odf.nxdl.xml +++ /dev/null @@ -1,171 +0,0 @@ - - - - - - - - Number of pixel per varphi section plot along the varphi_one fastest direction. - - - - - Number of pixel per varphi section plot along the capital_phi slow direction. - - - - - Number of pixel per varphi section plot along the varphi_two slowest direction. - - - - - Number of local maxima evaluated in the component analysis. - - - - - Base class to store an orientation distribution function (ODF) computation. - - - - Details about the algorithm used for computing the ODF. - - - - Point group of the crystal structure (International Table of Crystallography) - of the phase for which the here documented phase-dependent ODF was computed. - - - - - Point group assumed for processing-induced *sample symmetries*. - (according to International Table of Crystallography). - - - - - Halfwidth of the kernel. - - - - - Name of the kernel. - - - - - Resolution of the kernel. - - - - - - - Number of local maxima evaluated for the ODF. - - - - - - Euler angle representation of the kth-most maxima of the ODF - in decreasing order of the intensity maximum. - - - - - - - - - Disorientation threshold within which intensity of the ODF - is integrated for the component analysis. - - - - - Integrated ODF intensity within a theta-ball of SO3 about - each location as specified for each location in the order - and reported in the order of these locations. - - - - - - - - - Visualization of the ODF intensity as orthogonal sections through Euler space. - - This is one example of typical default plots used in the texture - community in Materials Engineering. - - Mind that the Euler space is a distorted space. Therefore, equivalent - orientations show intensity contributions in eventually multiple - locations. - - - - - ODF intensity at probed locations relative to - null model of a completely random texture. - - - - - - - - - - Pixel center angular position along the :math:`\varphi_1` direction. - - - - - - - - - Pixel center angular position along the :math:`\varphi_2` direction. - - - - - - - - - Pixel center angular position along the :math:`\Phi` direction. - - - - - - - - diff --git a/contributed_definitions/NXms_pf.nxdl.xml b/contributed_definitions/NXms_pf.nxdl.xml deleted file mode 100644 index 44db61d7b3..0000000000 --- a/contributed_definitions/NXms_pf.nxdl.xml +++ /dev/null @@ -1,111 +0,0 @@ - - - - - - - - Number of pixel per pole figure in the slow direction. - - - - - Number of pixel per pole figure in the fast direction. - - - - - Base class to store a pole figure (PF) computation. - - - - Details about the algorithm that was used to compute the pole figure. - - - - Point group of the crystal structure of the phase for which the - here documented phase-dependent pole figure was computed - (according to International Table of Crystallography). - - - - - Point group assumed for processing induced *sample symmetries* - (according to International Table of Crystallography). - - - - - Halfwidth of the kernel. - - - - - Miller indices (:math:`(hkl)[uvw]`) to specify the pole figure. - - - - - Resolution of the kernel. - - - - - - Pole figure. - - - - - Pole figure intensity. - - - - - - - - - Pixel center along y direction in the equatorial plane of - a stereographic projection of the unit sphere. - - - - - - - - - Pixel center along x direction in the equatorial plane of - a stereographic projection of the unit sphere. - - - - - - - - diff --git a/contributed_definitions/NXms_recon.nxdl.xml b/contributed_definitions/NXms_recon.nxdl.xml deleted file mode 100644 index f44d34239a..0000000000 --- a/contributed_definitions/NXms_recon.nxdl.xml +++ /dev/null @@ -1,454 +0,0 @@ - - - - - - - - The symbols used in the schema to specify e.g. dimensions of arrays. - - - - - The number of crystal projections. - - - - - The number of interface projections. - - - - - The number of assumed triple junction projections aka triple points. - - - - - - The number of crystals. - - - - - The number of interfaces - - - - - The number of triple lines - - - - - The number of quadruple junctions. - - - - - Base class to describe discretized (micro)structural features of a material. - - One instance of this base class can be used to describe the current configuration - the base class does not include time-dependent descriptions for the sake of - clarity and because of the fact that virtually all simulations or experiments - probe time by sampling. Therefore, time-dependent state descriptions should - be realized with creating a set of :ref:`NXms_snapshot_set` with instances of - :ref:`NXms_snapshot` using e.g. :ref:`NXms_recon` base class instances. - - - - - The configuration and parameterization of the reconstruction algorithm - whereby the microstructural features were identified. - - - - - Dimensionality of the analysis. - - This field can be used e.g. by a research data management system - to identify if the described feature set specifies a - one-, two-, or three-dimensional feature set. - - - - - - - - - - Which algorithm is used to reconstruct the features. - - - - - - - - - - - Threshold to define at which disorientation angle to assume - two crystalline regions have a significant orientation difference - which warrants to argue that there is an interface between the - two regions. - - - - - - - - - - - - - Projections of crystals on the sample surface as typically - characterized with optical or electron microscopy. - - - - Reference to lines(NXcg_polyline_set) which supports the - discretized shape of each cross-sectioned crystal. - - Most microscopy techniques support to generate only a two-dimensional - representation (projection) of the characterized material. - - For true volumetric techniques use the specifically - specialized crystals :ref:`NXms_feature_set` instead. - See stereology literature for more details e.g. - E.E. Underwood's book entitled Quantitative Stereology - - - - - Number of crystals. - - - - - Integer offset whereby the identifier of the first member - of the set differs from zero. - - Identifiers can be defined either implicitly or explicitly. - For implicit indexing identifiers are defined on the interval - :math:`[identifier_offset, identifier_offset + c - 1]`. - - - - - Identifier used for crystals for explicit indexing. - - - - - - - - How many phases are distinguished - - - - - Integer offset whereby the identifier of the first member - of the set differs from zero. - - - - - - Identifier used for phase for explicit indexing. - - - - - - - - - True, if the crystal makes contact with the edge of the ROI, - false otherwise. - - - - - - - - Average disorientation angle between individual orientation of the - crystal at probed positions (weighted by area of that position) versus - the average disorientation of the crystal. - - - - - - - - - Calibrated area of surrounded by the polyline about each crystal. - - - - - - - - - - Projections of grain or phase boundaries as typically sectioned - with optical or electron microscopy characterization. - - - - Reference to lines(NXcg_polyline_set) which supports the - discretized shape of each cross-sectioned crystal. - - Set of tuples of polyline segments which build the interface. - - - - - - Set of pairs of crystal_identifier resolved via depends_on which - are adjacent to each interface. - - - - - - - - The specific crystal_projections(NXms_feature_set) instance - to resolve crystal identifier. - - - - - - - Set of pairs of triple_point_identifier which the interface connects. - For 2D projections of 3D microstructural features a triple point is - physically only the projection of a triple line. - - - - - - - - The specific triple_line_projections(NXms_feature_set) instance - whereby to resolve triple_point identifier. - - - - - - - The length of the interface. - - This is not necessarily the same as the length of the individual - polyline segments whereby the interface is discretized. - - The actual coordinate system whereby the geometry is calibrated - with real physical dimensions is typically documented by the - depends_on attribute of the respective NXcg_primitive_set. - This depends_on attribute should point explicitly to an - instance of a :ref:`NXcoordinate_system` to support users as - much as possible with interpreting how and where the lines are - located in the reference frame. - - - - - - - - Integer offset whereby the identifier of the first member - of the set differs from zero. - - Identifiers can be defined either implicitly or explicitly. - For implicit indexing identifiers are defined on the interval - :math:`[identifier_offset, identifier_offset + c - 1]`. - - - - - Identifier for each interface using explicit indexing. - - - - - - - - - - Projections of triple lines as typically characterized with optical - or electron microscopy. - - Mind that most specimens are thermo-chemo-mechanically treated before - they are characterized. Therefore, the projected crystal defects are - have physically no longer the same structure as in the bulk. - - Examples are manifest as effects such as thermal grooving, or relaxation - effects of an intersection between a triple line that is cut - by the specimen surface as these defects are then exposed typically - to a different atmosphere and hence have different thermodynamic boundary - conditions than of their true volumetric defects in the bulk. - - - - Reference to points(NXcg_point_set) which supports the - locations of these triple points. - - - - - - Number of triple points. - - - - - Integer offset whereby the identifier of the first member - of the set differs from zero. - - Identifiers can be defined either implicitly or explicitly. - For implicit indexing identifiers are defined on the interval - :math:`[identifier_offset, identifier_offset + c - 1]`. - - - - - Identifier for each triple point using explicit indexing. - - - - - - - - Set of triple point identifiers. - - - - - - - The relevant points(NXcg_point_set) instance whereby to - resolve interface identifiers. - - - - - - Set of triplets of identifier of line-like features. - Each triplet resolves which three interface projections - the triple point connects. - - - - - - - - The specific interface_projections(NXms_feature_set) - instance whereby to resolve interface identifiers. - - - - - - - Triplet of identifier of polyline segments. Each triplet resolves - which three segments of polyline segments the triple junction connects. - - - - - - - - The specific lines(NXcg_polyline_set) instance to resolve - polyline segments. - - - - - - diff --git a/contributed_definitions/NXms_score_results.nxdl.xml b/contributed_definitions/NXms_score_results.nxdl.xml index 77c73b42f1..0e957612e5 100644 --- a/contributed_definitions/NXms_score_results.nxdl.xml +++ b/contributed_definitions/NXms_score_results.nxdl.xml @@ -2,9 +2,9 @@ +for is a matter of interpretation (fundamentally this is an assumption) and can differ for different descriptors--> The simulated or characterized material volume element aka domain. At least one instance of geometry required either NXcg_grid, @@ -693,27 +691,21 @@ electric_field(NXprocess): - + Specify if it was different from the number_of_processes in the NXcs_profiling super class. - + Specify if it was different from the number_of_threads in the NXcs_profiling super class. - + Specify if it was different from the number_of_threads in the NXcs_profiling super class. diff --git a/contributed_definitions/NXms_pf_set.nxdl.xml b/contributed_definitions/NXroi.nxdl.xml similarity index 72% rename from contributed_definitions/NXms_pf_set.nxdl.xml rename to contributed_definitions/NXroi.nxdl.xml index 14a4061d64..b77834bb69 100644 --- a/contributed_definitions/NXms_pf_set.nxdl.xml +++ b/contributed_definitions/NXroi.nxdl.xml @@ -2,9 +2,9 @@ - - + - Base class to group multiple :ref:`NXms_pf` instances. - - A collection of pole figure approximations. + Base class to describe a region-of-interest analyzed. - + + + Details about processing steps. + + + diff --git a/contributed_definitions/nyaml/NXcg_primitive_set.yaml b/contributed_definitions/nyaml/NXcg_primitive_set.yaml new file mode 100644 index 0000000000..586929596f --- /dev/null +++ b/contributed_definitions/nyaml/NXcg_primitive_set.yaml @@ -0,0 +1,136 @@ +category: base +doc: | + Computational geometry description of a set of primitives in Euclidean space. + + Primitives must neither be degenerated nor self-intersect. + Individual primitives can differ in their properties (e.g. size, shape, rotation). +# this base class defines common fields and properties of geometric primitives +# more complex primitive sets like NXcg_cylinder_set are considered specializations +# of NXcg_primitive_set. They contain all fields and groups which NXcg_primitive_set +# defines. This is an action of compositing an information set; an act of inheriting +# TODO:: many properties of non-degenerate primitives are in the number set +# R+ instead of in R+0 but currently NeXus does not allow for such value range +# constraints unless the coarsely discretized NX_INT, NX_POSINT, NX_FLOAT +# but there is no say NX_FLOAT+0 +# MK::but in computational geometry numerical precision matters as it defines +# whether objects numerically intersect or not and thus it can make a real difference +# if one stores triangles with 16, 32, or 64 bit precision, however: +# are two triangle_set instance A and B no longer conceptually triangle sets +# because A stores the positions of vertices using int8 while B stores such using float64 ? +# we here assume that we still conceptually talk that A and B are triangle sets +# but this brings at the level of the application definition the problem that if the +# precision is not properly constrainted a consuming application will not obtain +# the instances of the concept triangle_set with relevant high enough precision +# and thus neither the base class nor the application definition is specific enough +# for what it was designed in the first place - be specific about the requirements +# on your data... +symbols: + doc: | + The symbols used in the schema to specify e.g. dimensions of arrays. + d: | + The dimensionality of the space. + c: | + The cardinality of the set, i.e. the number of members. +type: group +NXcg_primitive_set(NXobject): + # individual specializations like NXcg_polyline_set typically overwrite + # the meaning of the depends_on concept to build consistent inference chains + # to enable an instantiation of the actual geometric primitives + \@depends_on(NX_CHAR): + doc: | + Hint to help resolve in which Euclidean coordinate system field values + like center or orientation are defined. + dimensionality(NX_POSINT): + doc: | + The dimensionality of the primitive set. + unit: NX_UNITLESS + enumeration: [1, 2, 3] + cardinality(NX_POSINT): + doc: | + The cardinality of the primitive set. + unit: NX_UNITLESS + identifier_offset(NX_INT): + doc: | + Integer offset whereby the identifier of the first member + of the set differs from zero. + + Identifiers can be defined either implicitly or explicitly. + For implicit indexing identifiers are defined on the interval + :math:`[identifier_offset, identifier_offset + c - 1]`. + + Therefore, implicit identifier are completely defined by the value of + identifier_offset and cardinality. For example if identifier run from + -2 to 3 the value for identifier_offset is -2. + + For explicit indexing the field identifier has to be used. + Fortran-/Matlab- and C-/Python-style indexing have specific implicit + identifier conventions where identifier_offset is 1 and 0 respectively. + unit: NX_UNITLESS + identifier(NX_INT): + doc: | + Identifier of each member for explicit indexing. + dim: (c,) # numpy style indexing + center(NX_NUMBER): + doc: | + The center of mass position of each primitive. + unit: NX_ANY + dim: (c, d) + # a depends_on to define in which coordinate system + is_center_of_mass(NX_BOOLEAN): + doc: | + True if the center is a center of mass. + dim: (c,) + shape(NX_NUMBER): + doc: | + A qualitative description of the shape of each primitive. + unit: NX_LENGTH + dim: (c, d) + length(NX_NUMBER): + doc: | + Qualifier for the length of characteristic features of the primitive. + + Often the term length is associated with the assumption that one + edge is parallel to an axis of the coordinate system. + unit: NX_LENGTH + dim: (c,) + width(NX_NUMBER): + doc: | + Qualifier often used to describe the length of one characteristic edge + within the coordinate system. + unit: NX_LENGTH + dim: (c,) + is_closed(NX_BOOLEAN): + doc: | + True if primitive is closed such that it has properties like area or volume. + dim: (c,) + volume(NX_NUMBER): + doc: | + Volume of each primitive. + + Set to NaN if does not apply for primitives for which is_closed is False. + unit: NX_VOLUME + dim: (c,) + area(NX_NUMBER): + doc: | + Alias for surface_area of each primitive. + + Set to NaN if does not apply for primitives for which is_closed is False. + unit: NX_AREA + dim: (c,) + orientation(NX_NUMBER): + doc: | + Direction unit vector which points along the + longest principal axis of each primitive. + + Use the depends_on attribute to specify in which coordinate system + these direction unit vectors are defined. + unit: NX_DIMENSIONLESS + dim: (c, d) + vertex_normal(NXcg_unit_normal_set): + edge_normal(NXcg_unit_normal_set): + face_normal(NXcg_unit_normal_set): + # roi(NXcg_parallelogram_set or NXcg_hexahedron_set) + # aabb(NXcg_parallelogram_set or NXcg_hexahedron_set) + # obb(NXcg_parallelogram_set or NXcg_hexahedron_set) + # MK::one could add (NXcg_parallelogram_set) and/or (NXcg_hexahedron_set) + # but then one would not give any hint at the base class level how to name diff --git a/contributed_definitions/nyaml/NXcoordinate_system.yaml b/contributed_definitions/nyaml/NXcoordinate_system.yaml new file mode 100644 index 0000000000..b219399001 --- /dev/null +++ b/contributed_definitions/nyaml/NXcoordinate_system.yaml @@ -0,0 +1,86 @@ +category: base +doc: | + Base class to detail a coordinate system (CS). + + Whenever possible, an instance of :ref:`NXcoordinate_system` should be used as + a member in an :ref:`NXcoordinate_system_set` and the name of the instance + should be this alias. This may support a process whereby jargon when talking + about coordinate systems and conventions may become cleaner for users + because it is not evident for people outside a lab that terms like e.g. + tip space or specimen space refer to the same coordinate system. + This is an example of jargon used in e.g. the field of atom + probe tomography. +type: group +NXcoordinate_system(NXobject): + origin(NX_CHAR): + doc: | + Human-readable field telling where the origin of this CS is. + Exemplar values could be *left corner of the lab bench*, *door-handle* + *pinhole through which the electron beam exists the pole piece*. + *barycenter of the triangle*, *center of mass of the stone*. + # implementing a proposal for "a common base table" along thoughts like: + # https://manual.nexusformat.org/classes/base_classes/NXtransformations.html#nxtransformations + # similar to a place where all transformations are stored + # https://www.zenodo.org/record/3526738/files/lyso009a_0087.JF07T32V01_master.h5?download=1 + alias(NX_CHAR): + doc: | + An alternative name given to that coordinate system. + type(NX_CHAR): + doc: | + Coordinate system type. + enumeration: [cartesian] + handedness(NX_CHAR): + doc: | + Handedness of the coordinate system if it is a Cartesian. + enumeration: [right_handed, left_handed] + x_alias(NX_CHAR): + doc: | + Possibility to define an alias for the name of the x-axis. + x_direction(NX_CHAR): + doc: | + Human-readable field telling in which direction the x-axis points if that + instance of :ref:`NXcoordinate_system` has no reference to any parent and as such + is the mighty world reference frame. + + Exemplar values could be direction of gravity. + x(NX_NUMBER): + doc: | + Base unit vector along the first axis which spans the coordinate system. + This axis is frequently referred to as the x-axis in real space and + the i-axis in reciprocal space. + unit: NX_LENGTH + dim: (3,) + y_alias(NX_CHAR): + doc: | + Possibility to define an alias for the name of the y-axis. + y_direction(NX_CHAR): + doc: | + Human-readable field telling in which direction the y-axis points if that + instance of :ref:`NXcoordinate_system` has no reference to any parent and as such + is the mighty world reference frame. + + See docstring of x_alias for further details. + y(NX_NUMBER): + doc: | + Base unit vector along the second axis which spans the coordinate system. + This axis is frequently referred to as the y-axis in real space and + the j-axis in reciprocal space. + unit: NX_LENGTH + dim: (3,) + z_alias(NX_CHAR): + doc: | + Possibility to define an alias for the name of the z-axis. + z_direction(NX_CHAR): + doc: | + Human-readable field telling in which direction the z-axis points if that + instance of :ref:`NXcoordinate_system` has no reference to any parent and as such + is the mighty world reference frame. + + See docstring of x_alias for further details. + z(NX_NUMBER): + doc: | + Base unit vector along the second axis which spans the coordinate system. + This axis is frequently referred to as the z-axis in real space and + the k-axis in reciprocal space. + unit: NX_LENGTH + dim: (3,) diff --git a/contributed_definitions/nyaml/NXcrystal_structure.yaml b/contributed_definitions/nyaml/NXcrystal_structure.yaml new file mode 100644 index 0000000000..796ac83d3c --- /dev/null +++ b/contributed_definitions/nyaml/NXcrystal_structure.yaml @@ -0,0 +1,178 @@ +category: base +doc: | + Base class to describe the atomic crystal structure of a phase. + + This base class contains key metadata that are relevant parameter to every + physics-based model to simulate radiation matter interaction. + + Examples where such base class is useful are kinematic or dynamic + diffraction simulations of e.g. (Kikuchi or other type of) patterns. +# The actual indexing of Kikuchi patterns may use different algorithms. +# Such are used within different workflows where simulated and measured +# Kikuchi pattern are compared to rate which phase and orientation is the most +# likely candidate describing the pattern measured at that each scan point +# respectively. If this evaluation yields scan points without any solutions, +# these are represented using the null-phase model phase0, aka n/a aka notIndexed. +# Traditionally, Hough transformation-based indexing has been the most frequently +# used algorithm. Dictionary-based alternatives are emerging. +symbols: + n_hkl: | + Number of reflectors (Miller crystallographic plane triplets). + n_pos: | + Number of atom positions. + d: | + Dimensionality of the lattice. +type: group +NXcrystal_structure(NXobject): + \@depends_on(NX_CHAR): + doc: | + Detail in which reference frame the unit cell is defined. + dimensionality(NX_POSINT): + doc: | + Dimensionality of the lattice. + enumeration: [1, 2, 3] + reference(NXidentifier): + doc: | + Reference to another resource that was used for + instantiating this structure model. + a_b_c(NX_NUMBER): + doc: | + Crystallography unit cell parameters a, b, and c. + unit: NX_LENGTH + dim: (d,) + # defined using which convention? + alpha_beta_gamma(NX_NUMBER): + doc: | + Crystallography unit cell parameters alpha, beta, and gamma. + unit: NX_ANGLE + dim: (d,) + area(NX_NUMBER): + doc: | + Area of the unit cell considering that d = 2. + unit: NX_AREA + volume(NX_NUMBER): + doc: | + Volume of the unit cell considering that d = 3. + unit: NX_VOLUME + crystal_system(NX_CHAR): + doc: | + Crystal system + enumeration: [triclinic, monoclinic, orthorhombic, tetragonal, rhombohedral, hexagonal, cubic] + # 2d + laue_group(NX_CHAR): + doc: | + Laue group using International Table of Crystallography Notation. + # add enumeration of all possible + point_group(NX_CHAR): + doc: | + Point group using International Table of Crystallography Notation. + # add enumeration all possible + # 3d + space_group(NX_CHAR): + doc: | + Space group from the International Table of Crystallography Notation. + # add enumeration of all possible + is_centrosymmetric(NX_BOOLEAN): + doc: | + True if space group is considered a centrosymmetric one. + False if space group is considered a non-centrosymmetric one. + Centrosymmetric has all types and combinations of symmetry elements + (translation, rotational axis, mirror planes, center of inversion) + Non-centrosymmetric compared to centrosymmetric is constrained (no inversion). + Chiral compared to non-centrosymmetric is constrained (no mirror planes). + is_chiral(NX_BOOLEAN): + doc: | + True if space group is considered a chiral one. + False if space group is consider a non-chiral one. + phase_identifier(NX_INT): + doc: | + Identifier for each phase. + + The value 0 is reserved for the unknown phase that represents the + null-model no sufficiently significant confirmation. In other words, + the phase_name is n/a, notIndexed. + + The phase identifier value has to match with the integer postfix of the + group name which represents that instance in a NeXus/HDF5 file, i.e. + if two phases were used e.g. 0 and 1, two instances of an + :ref:`NXcrystal_structure` named phase0 and phase1 + should be stored in the HDF5 file. + unit: NX_UNITLESS + # \@depends_on(NX_CHAR): + # doc: | + # Refers to the specific identifier_offset to consider. + # + # If not provided assume identifier_offset is 0. + phase_name(NX_CHAR): + doc: | + Name of the phase/alias. + + If the phase_identifier is 0 and one would like to use the field + phase_name the value should be n/a. + atom_identifier(NX_CHAR): + doc: | + Label for each atom position. + dim: (n_pos,) + atom_type(NX_UINT): + doc: | + The hash value :math:`H` is :math:`H = Z + N*256` with :math:`Z` + the number of protons and :math:`N` the number of neutrons + of each isotope respectively. Z and N have to be 8-bit unsigned integers. + For the rationale behind this `M. Kühbach et al. (2021) `_ + unit: NX_UNITLESS + dim: (n_pos,) + # atom_position(NXcg_point_set): + atom_position(NX_NUMBER): + doc: | + Atom positions. + dim: (n_pos, d) + unit: NX_ANY + \@depends_on(NX_CHAR): + doc: | + Reference to an instance of :ref:`NXcoordinate_system` + whereby the positions can be resolved. + # in addition we need to have a physical model e.g. kinematic or dynamical e-diffraction theory + # to describe the simulated Kikuchi pattern generated from such a model + atom_occupancy(NX_NUMBER): + doc: | + Relative occupancy of the atom position. + unit: NX_DIMENSIONLESS + dim: (n_pos,) + number_of_planes(NX_UINT): + doc: | + How many reflectors are distinguished. + + Value has to match value for symbol n_hkl. + unit: NX_UNITLESS + # Miller indices :math:`(hkl)[uvw]`. + miller(NX_NUMBER): + unit: NX_UNITLESS + doc: | + Miller indices :math:`(hkl)[uvw]` of the planes. + + The first triplet specify :math:`(hkl)` the second triplet :math:`[uvw]`. + Miller indices refer to the Cartesian right-handed coordinate system + of the unit cell. + dim: (n_hkl, 6) + dspacing(NX_NUMBER): + doc: | + Spacing between crystallographic planes as defined by field miller. + unit: NX_LENGTH + dim: (n_hkl,) + relative_intensity(NX_NUMBER): + doc: | + Relative intensity of the signal for the plane. + unit: NX_DIMENSIONLESS + dim: (n_hkl,) + number_of_scan_points(NX_UINT): + doc: | + In case the :ref:`NXcrystal_structure` base class is used + with analyzed orientation maps this field stores how many scan points + of the map were identified as that phase. + unit: NX_UNITLESS + ipfID(NXms_ipf): + pfID(NXms_pf): + odfID(NXms_odf): +# here the theoreticians expert (Marc deGraeff, Aimo Winkelmann, Peter Rez) +# can give some good suggestions on how to improve and ideally make even +# more general this section diff --git a/contributed_definitions/nyaml/NXem_base.yaml b/contributed_definitions/nyaml/NXem_base.yaml new file mode 100644 index 0000000000..2de8081a8c --- /dev/null +++ b/contributed_definitions/nyaml/NXem_base.yaml @@ -0,0 +1,297 @@ +category: base +# template to be used for an application definition +doc: | + Blue-print of a generic appdef for electron microscopy research formulated as a deep base class. + + This base class combines a method-specific and technical-design-level base class + instance to provide a template for storing parameterized descriptions of + pieces of information collected when performing electron microscopy research. + + The base class here shows all possible branches without making any statements + as to which of these have to be used in an instance. Thereby, the base class + provides a template how to name and structure concepts in a hierarchy + to support finding information and reducing the need for renaming and + restructuring information for a research field where many scientists perform + very specific research but who all also share commonalities like usage of + controlled electron beams, a focus on studies of electron beam matter interaction + to explore physical mechanisms and phenomena, or the desire to characterize materials + using electron microscopy. +# flesh out the description of that to read the docs, because currently the +# description on the NeXus front-page is overwhelming +# considering what we learned from the diataxis workshop we write here a +# specification neither a how to nor a tutorial which explains all the context +# because we address here developers of software +type: group +NXem_base(NXroot): + (NXprogram): + doc: | + A collection of all programs and libraries which are considered relevant + to understand with which software tools this NeXus file instance was + generated. Ideally, to enable a binary recreation from the input data. + + Examples include the name and version of the libraries used to write the + instance. Ideally, the software that writes these :ref:`NXprogram` instances + also includes the version of the set of NeXus classes i.e. the specific + set of base classes, application definitions, and contributed definitions + with which the here described concepts can be resolved. + + For the `pynxtools library `_ + which is used by the `NOMAD `_ + research data management system, it makes sense to store e.g. the GitHub + repository commit and respective submodule references used. + # each NeXus file instance should have a default plot + # however as there are cases when this cannot be assured we cannot + # make the default required, one example is e.g. a NeXus instance + # where scientists just store conventions without a default plot + cs_profiling(NXcs_profiling): + doc: | + The configuration of the I/O writer software (e.g. `pynxtools `_) + which was used to generate this NeXus file instance. + (NXentry): # means ENTRY(NXentry) + \@version(NX_CHAR): + doc: | + An at least as strong as SHA256 hashvalue of the file + which specifies the application definition. + definition(NX_CHAR): + doc: | + NeXus NXDL schema to which this file conforms. + enumeration: [NXem] + experiment_identifier(NXidentifier): + doc: | + Ideally, a (globally) unique persistent identifier + for referring to this experiment. + + An experiment should be understood in that this can be an experiment + in reality or a computer simulation because also the latter is an + experiment (see the Cambridge Dictionary experiment: + *a test done in order to find out something, eg if an idea is correct*). + + The identifier is usually issued by the facility, laboratory, + or the principle investigator. The identifier enables to link + experiments/simulations to e.g. proposals. + experiment_description(NX_CHAR): + doc: | + Free-text description about the experiment. + + Users are strongly advised to parameterize their description of the + experiment by using the respective base classes instead of writing prose + into this field. + + The reason is that such free-text field is difficult to machine-interpret. + The motivation behind keeping this field for now is to learn through + the information entered in this field in how far the current base + classes are incomplete. + start_time(NX_DATE_TIME): + doc: | + ISO 8601 time code with local time zone offset to UTC information included + when the microscope session started. If the application demands that time + codes in this section of the application definition should only be used + for specifying when the experiment was performed - and the exact + duration is not relevant - this start_time field should be used. + + Often though it is useful to specify a time interval via setting both + a start_time and an end_time because this enables software tools and + users to collect a more detailed bookkeeping of the experiment. + + The user should be aware that even with having both time instances specified, + it may not be possible to infer how long the experiment took or for how + long data were acquired. + + More detailed timing data over the course of the experiment have + to be collected to compute this. These computations can take + advantage of individual time stamps start_time and end_time + in :ref:`NXevent_data_em` instances. + end_time(NX_DATE_TIME): + doc: | + ISO 8601 time code with local time zone offset to UTC included when + the microscope session ended. See docstring of the start_time field + to see how the start_time and end_time should be used together. + (NXcite): + (NXprogram): + doc: | + The program and eventual software libraries used with which the + NeXus instance was created. For the NOMAD OASIS research data + management system e.g. pynxtools and eventually all modules + if desired. + # the above-description overwrites the default description of the NXprogram base class + # this is composed from the NXprogram base class + # program: + # \@version: + # \@url: + # NXnote and thumbnail dropped for the reason that these are + # arbitrary binary containers without any clear provenance. + (NXserialized): + doc: | + Possibility to store a collection of data artifacts + associated with the experiment. + # using NXserialized here instead of NXnote as the former is more specific + (NXuser): + doc: | + Contact information and eventually details of at least one person + who performed or was involved in the session. This can be the + principle investigator who performed this experiment or the student + who performed the simulation. + Adding multiple users if relevant is recommended. + name(NX_CHAR): + doc: | + Given (first) name and surname of the user. + affiliation(NX_CHAR): + doc: | + Name of the affiliation of the user at the point in time + when the experiment was performed. + address(NX_CHAR): + doc: | + Postal address of the affiliation. + email(NX_CHAR): + doc: | + Email address of the user at the point in time when the experiment + was performed. Writing the most permanently used email is recommended. + identifier(NXidentifier): + doc: | + Service as another mean of identification of a user than by the name. + Examples could be details about an ORCID or social media account of + the user. + telephone_number(NX_CHAR): + doc: | + (Business) (tele)phone number of the user at the point + in time when the experiment was performed. + role(NX_CHAR): + doc: | + Which role does the user have in the place and at the point + in time when the experiment was performed? Technician operating + the microscope, student, postdoc, principle investigator, or guest + are common examples. + sample(NXsample): + # NEW ISSUE: inject the conclusion from the discussion with Andrea + # according to SAMPLE.yaml 0f8df14 2022/06/15 + # ID: -> maps to name + # name: -> short_title + # user: -> not matched right now + # citation: doi ->why relevant, should be solved by RDMS + doc: | + A description of the material characterized in the experiment. + Sample and specimen are threaded as de facto synonyms. + Samples can be real specimens or virtual (see method). + method(NX_CHAR): + doc: | + A qualifier whether the sample is a real one or a + virtual one (in a computer simulation) + enumeration: [experiment, simulation] + # MK:: declared_by_vendor I would rather expect this for a substance + # COMPONENT.yaml + # SUBSTANCE: + # QUANTIFY + identifier(NXidentifier): + doc: | + Ideally, (globally) unique persistent identifier which distinguishes + the specimen from all others and especially the predecessor/origin + from where the specimen was cut. + + This field must not be used for an alias! Instead, use name. + + In cases where multiple specimens were loaded into the microscope, + the identifier has to resolve the specific sample, whose results are + stored by this :ref:`NXentry` instance, because a single NXentry should be + used only for the characterization of a single specimen. + + Details about the specimen preparation should be + stored in resources referring to parent_identifier. + parent_identifier(NXidentifier): + doc: | + Identifier of the sample from which the sample was cut or the string + *None*. The purpose of this field is to support functionalities + for tracking sample provenance via a research data management system. + preparation_date(NX_DATE_TIME): + doc: | + ISO 8601 time code with local time zone offset to UTC information + when the specimen was prepared. + + Ideally, report the end of the preparation, i.e. the last known time + the measured specimen surface was actively prepared. Ideally, this + matches the last timestamp that is mentioned in the digital resource + pointed to by parent_identifier. + + Knowing when the specimen was exposed to e.g. specific atmosphere is + especially required for environmentally sensitive material such as + hydrogen charged specimens or experiments including tracers with a + short half time. Additional time stamps prior to preparation_date + should better be placed in resources which describe but which do not pollute + the description here with prose. Resolving these connected pieces of information + is considered within the responsibility of the research data management + system. + name(NX_CHAR): + doc: | + An alias used to refer to the specimen to please readability for humans. + atom_types(NX_CHAR): + doc: | + List of comma-separated elements from the periodic table that are + contained in the sample. If the sample substance has multiple + components, all elements from each component must be included in + `atom_types`. + + The purpose of the field is to offer research data management systems an + opportunity to parse the relevant elements without having to interpret + these from the resources pointed to by parent_identifier or walk through + eventually deeply nested groups in data instances. + # NEW ISSUE: use Andrea and MarkusK groups for describing the geometry of the sample + thickness(NX_NUMBER): + doc: | + (Measured) sample thickness. + + The information is recorded to qualify if the beam used was likely + able to shine through the specimen. For scanning electron microscopy, + in many cases the specimen is typically thicker than what is + illuminatable by the electron beam. + + In this case the value should be set to the actual thickness of + the specimen viewed for an illumination situation where the nominal + surface normal of the specimen is parallel to the optical axis. + unit: NX_LENGTH + # \@units: nm + # NEW ISSUE: error estimates of the thickness and origin, i.e. how the value was obtained would be useful + # NEW ISSUE: error model + # NEW ISSUE: the KIT/SCC SEM, TEM schemata further qualify samples whether they are conductive e/ibeam sensitive + # etc. The problem with this is that beam sensitivity is too vague but spatiotemporal electron dose integral dependent + # KIT/SCC distinguish further conductivity and magnetic properties. While the motivation is clear, making + # it thus simple is likely problematic when the data entered in such fields remaining qualitative. + # what are good or bad properties, it would make sense though to quantify these values + # this includes the description of eventual plasma cleaning steps, + # just knowing that a sample was plasma cleaned is insufficient, maybe it was not cleaned long enough + # if plasma cleaning is done outside the EM than its certainly history, if it happens inside the EM + # are the ibeam description capabilities not sufficient enough? + density(NX_NUMBER): + # NX_MASS_PER_VOLUME + doc: | + (Measured) density of the specimen. + + For multi-layered specimens this field should only be used to describe + the density of the excited volume. For scanning electron microscopy + the usage of this field is discouraged and instead an instance of an + :ref:`NXinteraction_vol_em` within individual :ref:`NXevent_data_em` + instances can provide a cleaner description of the relevant details + why one may wish to store the density of the specimen. + unit: NX_ANY + description: + doc: | + Discouraged free-text field to provide further detail although adding + parent_identifier and having a working research data management system + should provide this contextualization. + # (NXmonitor): + (NXdata): + (NXcoordinate_system_set): + # link to an instance of an NXinstrument but that is anyway specialized for EM + measurement(NXem_msr): + simulation(NXem_sim): + (NXroi): + doc: | + A region-of-interest analyzed either during or after the session + for which specific processed data generated from the measured or the + simulated data are available. + se(NXem_img): + bse(NXem_img): + ebsd(NXem_ebsd): + eds(NXem_eds): + adf(NXem_adf): + eels(NXem_eels): + correlation(NXem_correlation): + # cl(NXem_cl): diff --git a/contributed_definitions/nyaml/NXem_correlation.yaml b/contributed_definitions/nyaml/NXem_correlation.yaml new file mode 100644 index 0000000000..2acf65ec86 --- /dev/null +++ b/contributed_definitions/nyaml/NXem_correlation.yaml @@ -0,0 +1,191 @@ +category: base +doc: | + Base class to combine different method-specific data in electron microscopy. + + This base class represent a template for documenting correlations + (spatial, temporal) between different method-specific results. +type: group +NXem_correlation(NXem_method): + (NXprocess): + doc: | + Details about processing steps. + sequence_index(NX_INT): + indexing(NXprocess): + doc: | + Details about correlated or logically connected EBSD datasets. + + One important class of such correlated experiments are the so-called + (quasi) in-situ experiments. In this case the same or nearly the same ROI + gets analyzed via a repetitive sequence of thermomechanical treatment, + sample preparation, measurement, on-the-fly-indexing. Phenomena + investigated are recrystallization, strain accumulation, material damage. + Post-processing is required to correlate and reidentify eventual + microstructural features or local ROIs across several orientation maps. + + Another important class of correlated experiments are the so-called + serial-sectioning experiments. Here the same sample is measured + repetitively after polishing each time, to create a stack of + orientation data which can be reconstructed to a + three-dimensional volume ROI. + + Data can be correlated in time, position (spatial), or both (spatiotemporal). + + Spatial correlations between repetitively characterized regions-of-interests + are typically correlated using image registration and alignment algorithms. + For this typically so-called landmarks are used. These can be grains with + a very large size or specific shape, i.e. grains which are qualitatively + different enough to be used as a guide how images are shifted relative to + one another. Other commonly used landmarks are fiducial marks which are + milled into the specimen surface using focus-ion beam milling and/or various + types of indentation methods. + + As far as the same physical region-of-interest is just measured several times, + the additional issue of the depth increment is not a concern. However, correct + assumptions for the depth increment, amount of material removed along the milling + direction is relevant for accurate and precise three-dimensional (serial-sectioning) + correlations. For these studies it can be tricky though to assume or estimate + useful depth increments. Different strategies have been proposed like + calibrations, wedged-shaped landmarks and computer simulation assisted + assumption making. + + Despite the use of landmarks, there are many practical issues which make the + processing of correlations imprecise and inaccurate. Among these are drift + and shift of the specimen, instabilities of the holder, the beam, irrespective + of the source of the drift, charging effects, here specifically causing local + image distortions and rotations which may require special processing algorithms + to reduce such imprecisions. + + Time correlations face all of the above-mentioned issues surplus the challenge + that specific experimental protocols have to be used to ensure the material state + is observed at specific physical time. The example of quasi in-situ characterization + of crystal growth phenomena, a common topic in engineering or modern catalysis research + makes it necessary to consider that e.g. the target value for the desired annealing + temperature is not just gauged based on macroscopic arguments but considers + that transient effects take place. Heating or quenching a sample might thus might + not have been executed under conditions in the interaction volume as they are + documented and/or assumed. + + These issue cause that correlations have an error margin as to how accurately + respective datasets were not only just synced based on the geometry of the + region-of-interests and the time markers but also to asssure which physical + conditions the specimen experienced over the course of the measurements. + + The fourth example of the em_om reference implementation explores the use of the + correlation group with a serial-sectioning datasets that was collected by the + classical Inconel 100 dataset collected by M. D. Uchic and colleagues + (M. Groeber M, Haley BK, Uchic MD, Dimiduk DM, Ghosh S 3d reconstruction and + characterization of polycrystalline microstructures using a fib-sem system data set. + Mater Charac 2006, 57 259–273. 10.1016/j.matchar.2006.01.019M). + + This dataset was specifically relevant in driving forward the implementation + of the DREAM.3D software. DREAM.3D is an open-source software project for + post-processing and reconstructing, i.e. correlating sets of orientation + microscopy data foremost spatially. One focus of the software is the + (post-)processing of EBSD datasets. Another cutting edge tool with similar + scope but a commercial solution by Bruker is QUBE which was developed by + P. Konijnenberg and coworkers. + + Conceptually, software like DREAM.3D supports users with creating linear + workflows of post-processing tasks. Workflows can be instructed via the + graphical user interface or via so-called pipeline processing via command line + calls. DREAM.3D is especially useful because its internal system documents all + input, output, and parameter of the processing steps. This makes DREAM.3D a + good candidate to interface with tools like em_om parser. Specifically, DREAM.3D + documents numerical results via a customized HDF5 file format called DREAM3D. + Workflow steps and settings are stored as nested dictionaries in JSON syntax + inside a supplementary JSON file or alongside the data in the DREAM3D file. + DREAM.3D has a few hundred algorithms implemented. These are called filters + in DREAM.3D terminology. + + Users configure a workflow which instructs DREAM.3D to send the data through + a chain of predefined and configured filters. Given that for each analysis + the filter is documented via its version tags surplus its parameter and setting + via a controlled vocabulary, interpreting the content of a DREAM3D HDF5 file + is possible in an automated manner using a parser. This makes DREAM.3D analyses + repeatable and self-descriptive. A key limitation though is that most frequently + the initial set of input data come from commercial files like ANG. + This missing link between the provenance of these input files, their associated + creation as electron microscope session, is also what NXem_ebsd solves. + + Nevertheless, as this can be solved with e.g. NXem_ebsd we are convinced that + the DREAM.3D and the em_om parser can work productively together to realize + RDMS-agnostic parsing of serial-section analyses. + + The internal documentation of the DREAM.3D workflow also simplifies the + provenance tracking represented by an instance of NXem_ebsd as not every + intermediate results has to be stored. Therefore, the fourth example + focuses on the key result obtained from DREAM.3D - the reconstructed + and aligned three-dimensional orientation map. + + Usually, this result is the starting point for further post-processing + and characterization of structural features. As here orientation microscopy + is insofar scale invariant using DREAM.3D, NXem_ebsd, and em_om should + be useful for different characterization methods, such as EBSD, Transmission + Kikuchi Diffraction (TKD), Automated Crystal Orientation Mapping (ACOM), + Nanobeam Electron Diffraction (using commercial systems like NanoMegas ASTAR) + or open-source implementations of these techniques (such as via pyxem/orix). + + The result of orientation microscopy methods are maps of local orientation + and thermodynamic phase (crystal structure) pieces of information. Virtually + all post-processing of such results for structural features includes again + a workflow of steps which are covered though by the NXms partner application + definition. The respective source of the data in an instance of NXms can + again be a link or reference to an instance of NXem_ebsd to complete the + chain of provenance. + (NXcrystal_structure): + roi(NXdata): + descriptor: + doc: | + Descriptor representing the image contrast. + # \@signal: # data + # \@axes: # [axis_y, axis_x] + # \@axis_x_indices: 0 + # \@axis_y_indices: 1 + # \@signal: + # \@axes: + # \@AXISNAME_indices: + # \@long_name: + title: + doc: | + Title of the default plot. + data(NX_NUMBER): + unit: NX_UNITLESS + doc: | + Descriptor values displaying the ROI. + dim: (n_z, n_y, n_x) + # n_0 slowest 3, n_1 slow 2, n_2 fast 1, rgb triplet is fastest 0 + # in axes fast to fastest + # while for _indices fastest to fast + \@long_name: + doc: | + Descriptor values. + axis_z(NX_NUMBER): + unit: NX_LENGTH + doc: | + Calibrated coordinate along the z-axis. + dim: (n_z,) + \@long_name: + doc: | + Label for the z axis + axis_y(NX_NUMBER): + unit: NX_LENGTH + doc: | + Calibrated coordinate along the y-axis. + dim: (n_y,) + \@long_name: + doc: | + Label for the y axis + axis_x(NX_NUMBER): + unit: NX_LENGTH + doc: | + Calibrated coordinate along the x-axis. + dim: (n_x,) + \@long_name: + doc: | + Label for the x axis + # NEW ISSUE: implement support for filters eventually many of them + # NEW ISSUE: for now only show that data from DREAM3D can be loaded. + # NEW ISSUE: how to handle landmarks + # NEW ISSUE: again an entire set of workflows such as rigid or non-rigid + # image registration etc. + # sequence_index(N0): diff --git a/contributed_definitions/nyaml/NXem_eds.yaml b/contributed_definitions/nyaml/NXem_eds.yaml new file mode 100644 index 0000000000..9a5a97fcd0 --- /dev/null +++ b/contributed_definitions/nyaml/NXem_eds.yaml @@ -0,0 +1,80 @@ +category: base +doc: | + Base class method-specific for energy-dispersive X-ray spectroscopy (EDS/EDX). + + `IUPAC instead of Siegbahn notation `_ should be used. + +# NEW ISSUE: use computational geometry to offer arbitrary scan pattern +# NEW ISSUE: make the binning flexible per scan point +symbols: + # n_p: Number of scan points + n_y: | + Number of pixel along the y direction, the slow direction + n_x: | + Number of pixel along the x direction, the fast direction + n_photon_energy: | + Number of X-ray photon energy (bins), the fastest direction. + n_elements: | + Number of identified elements + n_peaks: | + Number of peaks detected +type: group +NXem_eds(NXem_method): + # NXprocess is composed from NXem_method base class + # SPECTRUM_SET(NXspectrum_set) is composed from NXem_method base class + # for post-processing of/with the above-defined data entries + # including feedback from Christoph Pauly (from MX Uni Saarland, NFDI-MatWerk), + # Sabine Bergmann and Sebastian Brückner (both from FAIRmat, IKZ), + # and Adrien Teutrie, Cecile Hebert (EPFL) + indexing(NXprocess): + doc: | + Details about computational steps how peaks were indexed as elements. + (NXprogram): + doc: | + The program with which the indexing was performed. + PEAK(NXpeak): + doc: | + Name and location of each X-ray line which was indexed as a known ion. + For each ion, an NXion instance should be created which specifies + the origin of the signal. For each ion also the relevant IUPAC notation + X-ray lines should be specified. + (NXion): + energy_range(NX_NUMBER): + doc: | + Associated lower :math:`[e_{min}, e_{max}]` bounds of the + energy which is assumed associated with this peak. + unit: NX_ENERGY + dim: (2,) + energy(NX_NUMBER): + doc: | + Theoretical energy of the line according to IUPAC. + unit: NX_ENERGY + iupac_line_names(NX_CHAR): + doc: | + IUPAC notation identifier of the line which the peak represents. + + This can be a list of IUPAC notations for (the seldom) case that + multiple lines are grouped with the same peak. + dim: (i,) + element_names(NX_CHAR): + doc: | + List of the names of identified elements. + dim: (n_elements,) + IMAGE_R_SET(NXimage_r_set): + doc: | + Individual element-specific EDS/EDX/EDXS/SXES mapping + + A composition map is an image whose intensities for each pixel are the + accumulated X-ray quanta *under the curve(s)* of a set of peaks. + + These element-specific EDS maps are NXimage_r_set instances + and need to be named with the name of the element from the + element_names field. + (NXprocess): + peaks(NX_CHAR): + doc: | + A list of NXpeak instance names whose X-ray quanta + where accumulated for each pixel which yields an element-specific + EDS map. + dim: (n_peaks,) + # everything else is composed from NXimage_r_set diff --git a/contributed_definitions/nyaml/NXem_eels.yaml b/contributed_definitions/nyaml/NXem_eels.yaml new file mode 100644 index 0000000000..4e5c69fe4a --- /dev/null +++ b/contributed_definitions/nyaml/NXem_eels.yaml @@ -0,0 +1,39 @@ +category: base +doc: | + Base class method-specific for Electron Energy Loss Spectroscopy (EELS). +symbols: + n_energy_loss: | + Number of electron energy loss bins. +type: group +NXem_eels(NXem_method): + # NXem_method can offers to keep a SPECTRUM_SET + # NXem_method also has an NXprocess which in this base class can be + # specialized to include EELS-specific post-processing + SPECTRUM_SET(NXspectrum_set): + doc: | + NXspectrum_set_em specialized for EELS. + stack(NXdata): + # \@signal: data_counts + # \@axes: [axis_y, axis_x, axis_energy_loss] + # \@energy_loss_indices: 2 + # \@axis_x_indices: 1 + # \@axis_y_indices: 0 + axis_energy_loss(NX_NUMBER): + unit: NX_ENERGY + dim: (n_energy_loss,) + \@long_name(NX_CHAR): + doc: | + Energy loss. + summary(NXdata): + # \@signal: data_counts + # \@axes: [axis_energy_loss] + # \@energy_loss_indices: 0 + axis_energy_loss(NX_NUMBER): + unit: NX_ENERGY + doc: | + Energy loss. + dim: (n_energy_loss,) + \@long_name(NX_CHAR): + doc: | + Energy loss. + # collection if needed can be composed from NXspectrum_set diff --git a/contributed_definitions/nyaml/NXem_img.yaml b/contributed_definitions/nyaml/NXem_img.yaml new file mode 100644 index 0000000000..8c257d121a --- /dev/null +++ b/contributed_definitions/nyaml/NXem_img.yaml @@ -0,0 +1,25 @@ +category: base +doc: | + Base class for method-specific generic imaging. + + In the majority of cases simple d-dimensional regular scan patterns are used + to probe a region-of-interest (ROI). Examples can be single point aka spot + measurements, line profiles, or (rectangular) surface mappings. + The latter pattern is the most frequently used. + + For now the base class provides for scans for which the settings, + binning, and energy resolution is the same for each scan point. +symbols: + n_images: | + Number of images in the stack. + n_y: | + Number of pixel per image in the slow direction. + n_x: | + Number of pixel per image in the fast direction. +type: group +NXem_img(NXem_method): + imaging_mode(NX_CHAR): + doc: | + Which imaging mode was used? + enumeration: [secondary_electron, backscattered_electron] + IMAGE_R_SET(NXimage_r_set): diff --git a/contributed_definitions/nyaml/NXem_method.yaml b/contributed_definitions/nyaml/NXem_method.yaml new file mode 100644 index 0000000000..afe91de973 --- /dev/null +++ b/contributed_definitions/nyaml/NXem_method.yaml @@ -0,0 +1,21 @@ +category: base +doc: | + Base class to describe specific analysis methods in electron microscopy. + + This base class represent a template how specialized, deep, and method-specific + base classes can be defined with which an (NXem) application + definition gets equipped with specific groups to document method-specific + processing steps and report analyzed quantities. + + The template base class name :ref:`NXem_method` needs to be changed for each + method e.g. :ref:`NXem_adf`, :ref:`NXem_ebsd`, :ref:`NXem_eels`, :ref:`NXem_eds`. +# :ref:`NXem_se`, :ref:`NXem_bse`. +type: group +NXem_method(NXobject): + (NXprocess): + doc: | + Details about processing steps. + sequence_index(NX_INT): + IMAGE_R_SET(NXimage_r_set): + IMAGE_C_SET(NXimage_c_set): + SPECTRUM_SET(NXspectrum_set): diff --git a/contributed_definitions/nyaml/NXem_msr.yaml b/contributed_definitions/nyaml/NXem_msr.yaml new file mode 100644 index 0000000000..16c349ca58 --- /dev/null +++ b/contributed_definitions/nyaml/NXem_msr.yaml @@ -0,0 +1,63 @@ +category: base +doc: | + Base class for collecting a session with a real electron microscope. + + For collecting data and experiments which are simulations of an + electron microscope use the :ref:`NXem_sim` base class. +type: group +NXem_msr(NXem_method): + em_lab(NXinstrument): + doc: | + (Meta)data of the microscope and the lab in which it stands. + + This em_lab group differs from potential em_lab groups inside + :ref:`NXevent_data_em` instances in that here the more static descriptions + are kept while changing, i.e. time-dependent pieces of information are + logged, via the em_lab group inside the desired number of instances + of NXevent_data_em. + + While using an :ref:`NXevent_data_em` instance, users should store only those + settings about a component which are relevant to understand the current + state of the component. Here, current means for the time interval which + the event covers (as it is detailed via start_time and end_time) timestamps. + + For example it is not relevant to store in each :ref:`NXevent_data_em` + electron_source group again the details of the gun type and the manufacturer + but only the high-voltage value and that only if it is different from the value + that is specified in the em_lab section for the static settings. + + In effect, this defines an information inference hierarchy which starts + in an individual :ref:`NXevent_data_em` instance followed by a probing of the + static section. + instrument_name(NX_CHAR): + doc: | + Given name of the microscope at the hosting institution. + This is an alias. Examples could be NionHermes, Titan, JEOL, + Gemini, etc. + location(NX_CHAR): + doc: | + Location of the lab or place where the instrument is installed. + Using GEOREF is preferred. + (NXfabrication): + (NXchamber): + (NXebeam_column): + (NXibeam_column): + (NXoptical_system_em): + (NXdetector): + doc: | + Description of the type of the detector. + + Electron microscopes have typically multiple detectors. + Different technologies are in use like CCD, scintillator, + direct electron, CMOS, or image plate to name but a few. + local_name(NX_CHAR): + doc: | + Instrument-specific alias/name + # it is unfortunate that for NXdetector there are already many places + # how one can specify details which could equally end up in fabrications + # we should give better guidance which option to use, pr to niac + # (NXfabrication): + (NXpump): + (NXstage_lab): + (NXevent_data_em_set): +# (NXevent_data_em): diff --git a/contributed_definitions/nyaml/NXem_sim.yaml b/contributed_definitions/nyaml/NXem_sim.yaml new file mode 100644 index 0000000000..81fe20db1b --- /dev/null +++ b/contributed_definitions/nyaml/NXem_sim.yaml @@ -0,0 +1,34 @@ +category: base +doc: | + Base class for simulating electron microscopy relevant beam-matter interaction. + + The concept behind this base class is to keep it as generic as possible + that simulations of electron/ion beam interaction with matter can be + represented. This base class is envisioned as the twin of the :ref:`NXem_msr` + base class. + + It is an attempt to test the idea if at some point one might even use the + same base class template to describe measurements and computer simulations + of electron microscopy. This idea is attractive because the only practical + difference between a description of a measurement with a microscope and a + computer simulation is that the latter is typically a substantially simplified + representation of the real microscope surplus the focus of the research + in such cases on specific questions. + + Such simplification can be with respect to the optical setup, typically the + ignoring of the fact that the electron beam is produced by a complex setup + of lenses while in simulations often single Einzel lenses are considered. + Dynamics of the environment like temperature fluctuation in a lab, vibrations, + users, and multiple detectors are typically either ignored or reduced in + complexity and number and coming with idealizations to keep the simulations + focused on the specific reason questions and efficiently numerically executable. +# abTEM and other simulation packages, TEMgym, etc. +type: group +NXem_sim(NXem_method): + simulation(NXprocess): + doc: | + Details about the simulation. + # sequence_index(N): + (NXprogram): + (NXcg_geodesic_mesh): + (NXimage_r_set_diff): diff --git a/contributed_definitions/nyaml/NXroi.yaml b/contributed_definitions/nyaml/NXroi.yaml new file mode 100644 index 0000000000..a8ba2426ad --- /dev/null +++ b/contributed_definitions/nyaml/NXroi.yaml @@ -0,0 +1,9 @@ +category: base +doc: | + Base class to describe a region-of-interest analyzed. +type: group +NXroi(NXobject): + (NXprocess): + doc: | + Details about processing steps. + sequence_index(NX_INT):