Initial H5OINAReader. Needs a special build of EbsdLib to fully work.

Signed-off-by: Michael Jackson <[email protected]>

CMakeLists.txt

@@ -53,7 +53,7 @@ enable_vcpkg_manifest_feature(TEST_VAR COMPLEX_ENABLE_ITKImageProcessing FEATURE
 # Is the OrientationAnalysis Plugin enabled [DEFAULT=ON]
 # --------------------------------------------------------------------------------------------------
 option(COMPLEX_ENABLE_OrientationAnalysis "Enable the OrientationAnalysis Plugin" ON)
-enable_vcpkg_manifest_feature(TEST_VAR COMPLEX_ENABLE_OrientationAnalysis FEATURE "ebsd")
+#enable_vcpkg_manifest_feature(TEST_VAR COMPLEX_ENABLE_OrientationAnalysis FEATURE "ebsd")
 
 option(COMPLEX_CONDA_BUILD "Enables conda-build specific layout" OFF)
 

src/Plugins/ComplexCore/src/ComplexCore/Filters/ImportHDF5Dataset.cpp

@@ -126,10 +126,6 @@ IFilter::PreflightResult ImportHDF5Dataset::preflightImpl(const DataStructure& d
 
   fs::path inputFilePath(inputFile);
   std::string ext = inputFilePath.extension().string();
-  if(ext != ".h5" && ext != ".hdf5" && ext != ".dream3d")
-  {
-    return {nonstd::make_unexpected(std::vector<Error>{Error{-20002, fmt::format("The selected file '{}' is not an HDF5 file.", inputFilePath.filename().string())}})};
-  }
 
   if(!fs::exists(inputFilePath))
   {

src/Plugins/OrientationAnalysis/CMakeLists.txt

@@ -1,9 +1,12 @@
 # ------------------------------------------------------------------------------
 # Required EbsdLib and H5Support
 # ------------------------------------------------------------------------------
-find_package(H5Support REQUIRED)
+#find_package(H5Support REQUIRED)
 
-find_package(EbsdLib REQUIRED)
+#find_package(EbsdLib REQUIRED)
+
+add_subdirectory(${complex_SOURCE_DIR}/../H5Support ${complex_BINARY_DIR}/H5Support)
+add_subdirectory(${complex_SOURCE_DIR}/../EbsdLib ${complex_BINARY_DIR}/EbsdLib)
 
 # ------------------------------------------------------------------------------
 # EbsdLib needs install rules for creating packages
@@ -63,6 +66,7 @@ set(FilterList
   GenerateQuaternionConjugateFilter
   ImportH5EspritDataFilter
   ImportH5OimDataFilter
+  ImportH5OinaDataFilter
   INLWriterFilter
   MergeTwinsFilter
   NeighborOrientationCorrelationFilter
@@ -196,6 +200,7 @@ set(filter_algorithms
   GroupFeatures
   ImportH5EspritData
   ImportH5OimData
+  ImportH5OinaData
   INLWriter
   MergeTwins
   NeighborOrientationCorrelation

src/Plugins/OrientationAnalysis/docs/ImportH5OinaDataFilter.md

@@ -0,0 +1,96 @@
+# Import Oxford AZtec H5OINA Data File (.h5oina)
+
+## Group (Subgroup)
+
+Import/Export (Import)
+
+## Description
+
+This filter will read data from a single .h5oina file into a new **Image Geometry**, allowing the immediate use of
+**Filters** on the data instead of having to generate the intermediate .h5ebsd file. A **Cell Attribute Matrix** and
+**Ensemble Attribute Matrix** will also be created to hold the imported EBSD information. Currently, the user has no control
+over the names of the created **Attribute Arrays**.
+
+### Limitations of the Filter
+
+The current implementation only understands the FORMAT VERSION 2.0 of the H5OINA file. This means that a user
+can use a newer H5OINA file but the filter will only extract out the VERSION 2.0 headers and data. If the user
+needs additional data from the file, the "Import HDF5 Dataset" filter can be used to agment this filter.
+
+![Overview of the user interface.](Images/ImportH5OinaFilter_1.png)
+
+## Notes About Reference Frames
+
+In order to bring the crystal reference frame and sample reference frame into coincidence, rotations **MAY** need to be applied to the data. There are 2 filters that can perform the necessary rotations.
+
++ [Rotate Euler Reference Frame](../RotateEulerRefFrameFilter/index.html)
++ [Rotate Sample Reference Frame](../RotateSampleRefFrameFilter/index.html)
+
+Historical reference frame operations for a .ctf file are the following:
+
++ Sample Reference Frame: 180<sup>o</sup> about the <010> Axis
++ Crystal Reference Frame: None
+
+The user also may want to assign un-indexed pixels to be ignored by flagging them as "bad". The Threshold Objects **Filter** can be used to define this *mask* by thresholding on values such as *Error* = 0.
+
+### Radians and Degrees
+
+All orientation data in the H5OINA file are in radians.
+
+### The Axis Alignment Issue for Hexagonal Symmetry [1]
+
++ The issue with hexagonal materials is the alignment of the Cartesian coordinate system used for calculations with the crystal coordinate system (the Bravais lattice).
++ In one convention (e.g. EDAX.TSL), the x-axis, i.e. [1,0,0], is aligned with the crystal a1 axis, i.e. the [2,-1,-1,0] direction. In this case, the y-axis is aligned with the [0,1,-1,0] direction. (Green Axis in Figure 1)
++ In the other convention, (e.g. Oxford Instr, Univ. Metz software), the x-axis, i.e. [1,0,0], is aligned with the crystal [1,0,-1,0] direction. In this case, the y-axis is aligned with the [-1,2,-1,0] direction. (Red Axis in Figure 1)
++ This is important because texture analysis can lead to an ambiguity as to the alignment of [2,-1,-1,0] versus [1,0,-1,0], with apparent **30 Degree** shifts in the data.
++ Caution: it appears that the axis alignment is a choice that must be made when installing TSL software so determination of which convention is in use must be made on a case-by-case basis. It is fixed to the y-convention in the HKL software.
++ The main clue that something is wrong in a conversion is that either the 2110 & 1010 pole figures are transposed, or that a peak in the inverse pole figure that should be present at 2110 has shifted over to 1010.
++ DREAM.3D uses the TSL/EDAX convention.
++ __The result of this is that the filter will by default add 30 degrees to the second Euler Angle (phi2) when reading Oxford Instr (.ctf) files. This can be disabled by the user if necessary.__
+
+| Figure 1 |
+|--------|
+| ![Figure showing 30 Degree conversions](Images/Hexagonal_Axis_Alignment.png) |
+| **Figure 1:** showing TSL and Oxford Instr. conventions. EDAX/TSL is in **Green**. Oxford Inst. is in **Red** |
+
+## Parameters
+
+| Name | Type | Description |
+|------|------| ----------- |
+| Input File | File Path | The input .h5 file path |
+| Scan Name | String | The name of the scan in the .h5oina file. |
+| Z Spacing | float | The spacing in microns between each layer. |
+| Origin | float (3x1) | The origin of the volume |
+| Import Pattern Data | bool | Default=OFF |
+| Hexagonal Axis Alignment | bool | Should the filter convert a Hexagonal phase to the EDAX standard for x-axis alignment |
+| Convert Phase data to Int32 | bool | Should the phase data be converted to Int32 or keep the original uint8 |
+
+## Created Objects
+
+| Kind        | Default Name | Type | Comp Dims | Description                            |
+|-------------|--------------|------|-----------|----------------------------------------|
+| Data Container  | ImageDataContainer | N/A | N/A    | Created Data Container name with an **Image Geometry** |
+| Attribute Matrix  | CellData | Cell | N/A    | Created **Cell Attribute Matrix** name  |
+| Attribute Matrix  | CellEnsembleData | Cell Ensemble | N/A    | Created **Cell Ensemble Attribute Matrix** name  |
+| Cell Attribute Array | Band Contrast  | uint8 | (1) |  |
+| Cell Attribute Array | Band Slope  | uint8 | (1) |  |
+| Cell Attribute Array | Bands  | uint8 | (1) |  |
+| Cell Attribute Array | Error  | uint8 | (1) | The error descriptions are saved as attributes in the .h5oina file  |
+| Cell Attribute Array | Euler  | float | (3) | Three angles defining the orientation of the **Cell** in Bunge convention (Z-X-Z)  |
+| Cell Attribute Array | MeanAngularDeviation  | float | (1) |  |
+| Cell Attribute Array | Phase  | uint8 | (1) |  |
+| Cell Attribute Array | X  | float | (1) | The X Position of the scan point |
+| Cell Attribute Array | Y  | float | (1) | The Y Position of the scan point |
+| Ensemble Attribute Array | CrystalStructures | uint32_t | (1) | Enumeration representing the crystal structure for each **Ensemble** |
+| Ensemble Attribute Array | LatticeConstants | float | (6) | The 6 values that define the lattice constants for each **Ensemble**|
+| Ensemble Attribute Array | MaterialName | String | (1) | Name of each **Ensemble** |
+
+## Example Pipelines
+
+## References
+
+[1] Rollett, A.D. Lecture Slides located at [http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf](http://pajarito.materials.cmu.edu/rollett/27750/L17-EBSD-analysis-31Mar16.pdf)
+
+## DREAM3DNX Help
+
+Check out our GitHub community page at [DREAM3DNX-Issues](https://github.com/BlueQuartzSoftware/DREAM3DNX-Issues) to report bugs, ask the community for help, discuss features, or get help from the developers.

src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ImportH5OinaData.cpp

@@ -0,0 +1,133 @@
+#include "ImportH5OinaData.hpp"
+
+#include "complex/DataStructure/DataArray.hpp"
+#include "complex/DataStructure/Geometry/ImageGeom.hpp"
+
+using namespace complex;
+
+namespace
+{
+
+template <typename T>
+void copyRawData(const ImportH5DataInputValues* m_InputValues, size_t totalPoints, DataStructure& m_DataStructure, H5OINAReader& m_Reader, const std::string& name, usize offset)
+{
+  using ArrayType = DataArray<T>;
+  auto& dataRef = m_DataStructure.getDataRefAs<ArrayType>(m_InputValues->CellAttributeMatrixPath.createChildPath(name));
+  auto* dataStorePtr = dataRef.getDataStore();
+
+  usize numElements = m_Reader.getNumberOfElements();
+  const nonstd::span<T> rawDataPtr(reinterpret_cast<T*>(m_Reader.getPointerByName(name)), numElements);
+  std::copy(rawDataPtr.begin(), rawDataPtr.end(), dataStorePtr->begin() + offset);
+}
+
+template <typename T>
+void convertHexEulerAngle(const ImportH5DataInputValues* m_InputValues, size_t totalPoints, DataStructure& m_DataStructure)
+{
+  using ArrayType = DataArray<T>;
+
+  if(m_InputValues->EdaxHexagonalAlignment)
+  {
+    auto& crystalStructuresRef = m_DataStructure.getDataRefAs<UInt32Array>(m_InputValues->CellEnsembleAttributeMatrixPath.createChildPath(EbsdLib::AngFile::CrystalStructures));
+    auto& crystalStructuresDSRef = crystalStructuresRef.getDataStoreRef();
+
+    auto& cellPhasesRef = m_DataStructure.getDataRefAs<ArrayType>(m_InputValues->CellAttributeMatrixPath.createChildPath(EbsdLib::H5OINA::Phase));
+    auto& cellPhasesDSRef = cellPhasesRef.getDataStoreRef();
+
+    auto& eulerRef = m_DataStructure.getDataRefAs<Float32Array>(m_InputValues->CellAttributeMatrixPath.createChildPath(EbsdLib::H5OINA::Euler));
+    auto& eulerDataStoreRef = eulerRef.getDataStoreRef();
+
+    for(size_t i = 0; i < totalPoints; i++)
+    {
+      if(crystalStructuresDSRef[cellPhasesDSRef[i]] == EbsdLib::CrystalStructure::Hexagonal_High)
+      {
+        eulerDataStoreRef[3 * i + 2] = eulerDataStoreRef[3 * i + 2] + 30.0F; // See the documentation for this correction factor
+      }
+    }
+  }
+}
+
+} // namespace
+
+// -----------------------------------------------------------------------------
+ImportH5OinaData::ImportH5OinaData(DataStructure& dataStructure, const IFilter::MessageHandler& mesgHandler, const std::atomic_bool& shouldCancel, ImportH5DataInputValues* inputValues)
+: ImportH5Data<H5OINAReader>(dataStructure, mesgHandler, shouldCancel, inputValues)
+{
+}
+
+// -----------------------------------------------------------------------------
+ImportH5OinaData::~ImportH5OinaData() noexcept = default;
+
+// -----------------------------------------------------------------------------
+Result<> ImportH5OinaData::operator()()
+{
+  return execute();
+}
+
+// -----------------------------------------------------------------------------
+Result<> ImportH5OinaData::copyRawEbsdData(int index)
+{
+  const auto& imageGeom = m_DataStructure.getDataRefAs<ImageGeom>(m_InputValues->ImageGeometryPath);
+  const usize totalPoints = imageGeom.getNumXCells() * imageGeom.getNumYCells();
+  const usize offset = index * totalPoints;
+
+  copyRawData<uint8>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::BandContrast, offset);
+  copyRawData<uint8>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::BandSlope, offset);
+  copyRawData<uint8>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::Bands, offset);
+  copyRawData<uint8>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::Error, offset);
+  copyRawData<float>(m_InputValues, totalPoints * 3, m_DataStructure, *m_Reader, EbsdLib::H5OINA::Euler, offset);
+  copyRawData<float>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::MeanAngularDeviation, offset);
+  if(m_InputValues->ConvertPhaseToInt32)
+  {
+    const nonstd::span<uint8> rawDataPtr(reinterpret_cast<uint8*>(m_Reader->getPointerByName(EbsdLib::H5OINA::Phase)), totalPoints);
+    using ArrayType = DataArray<int32>;
+    auto& dataRef = m_DataStructure.getDataRefAs<ArrayType>(m_InputValues->CellAttributeMatrixPath.createChildPath(EbsdLib::H5OINA::Phase));
+    auto* dataStorePtr = dataRef.getDataStore();
+    for(size_t i = 0; i < totalPoints; i++)
+    {
+      dataStorePtr->setValue(i + offset, static_cast<int32>(rawDataPtr[i]));
+    }
+  }
+  else
+  {
+    copyRawData<uint8>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::Phase, offset);
+  }
+  copyRawData<float>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::X, offset);
+  copyRawData<float>(m_InputValues, totalPoints, m_DataStructure, *m_Reader, EbsdLib::H5OINA::Y, offset);
+
+  if(m_InputValues->EdaxHexagonalAlignment)
+  {
+    if(m_InputValues->ConvertPhaseToInt32)
+    {
+      convertHexEulerAngle<int32>(m_InputValues, totalPoints, m_DataStructure);
+    }
+    else
+    {
+      convertHexEulerAngle<uint8>(m_InputValues, totalPoints, m_DataStructure);
+    }
+  }
+
+
+  if(m_InputValues->ReadPatternData)
+  {
+    const uint16* patternDataPtr = m_Reader->getPatternData();
+    std::array<int32, 2> pDims = {{0, 0}};
+    m_Reader->getPatternDims(pDims);
+    if(pDims[0] != 0 && pDims[1] != 0)
+    {
+      std::vector<usize> pDimsV(2);
+      pDimsV[0] = pDims[0];
+      pDimsV[1] = pDims[1];
+      auto& patternData = m_DataStructure.getDataRefAs<UInt8Array>(m_InputValues->CellAttributeMatrixPath.createChildPath(EbsdLib::H5OINA::UnprocessedPatterns));
+      const usize numComponents = patternData.getNumberOfComponents();
+      for(usize i = 0; i < totalPoints; i++)
+      {
+        for(usize j = 0; j < numComponents; ++j)
+        {
+          patternData[offset + numComponents * i + j] = patternDataPtr[numComponents * i + j];
+        }
+      }
+    }
+  }
+
+  return {};
+}

src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/Algorithms/ImportH5OinaData.hpp

@@ -0,0 +1,32 @@
+#pragma once
+
+#include "OrientationAnalysis/OrientationAnalysis_export.hpp"
+#include "OrientationAnalysis/utilities/ImportH5Data.hpp"
+
+namespace complex
+{
+
+/**
+ * @class ImportH5OinaData
+ * @brief This filter will read a single .h5 file into a new Image Geometry, allowing the immediate use of Filters on the data instead of having to generate the
+ * intermediate .h5ebsd file.
+ */
+
+class ORIENTATIONANALYSIS_EXPORT ImportH5OinaData : public ImportH5Data<H5OINAReader>
+{
+public:
+  ImportH5OinaData(DataStructure& dataStructure, const IFilter::MessageHandler& mesgHandler, const std::atomic_bool& shouldCancel, ImportH5DataInputValues* inputValues);
+  ~ImportH5OinaData() noexcept override;
+
+  ImportH5OinaData(const ImportH5OinaData&) = delete;
+  ImportH5OinaData(ImportH5OinaData&&) noexcept = delete;
+  ImportH5OinaData& operator=(const ImportH5OinaData&) = delete;
+  ImportH5OinaData& operator=(ImportH5OinaData&&) noexcept = delete;
+
+  Result<> operator()();
+
+  Result<> copyRawEbsdData(int index) override;
+
+};
+
+} // namespace complex

src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/ImportH5EspritDataFilter.cpp

@@ -67,7 +67,7 @@ Parameters ImportH5EspritDataFilter::parameters() const
   params.insertSeparator(Parameters::Separator{"Input Parameters"});
   params.insert(std::make_unique<OEMEbsdScanSelectionParameter>(k_SelectedScanNames_Key, "Scan Names", "The name of the scan in the .h5 file. EDAX can store multiple scans in a single file",
                                                                 OEMEbsdScanSelectionParameter::ValueType{},
-                                                                OEMEbsdScanSelectionParameter::AllowedManufacturers{EbsdLib::OEM::Bruker, EbsdLib::OEM::DREAM3D},
+                                                                /* OEMEbsdScanSelectionParameter::AllowedManufacturers{EbsdLib::OEM::Bruker, EbsdLib::OEM::DREAM3D},*/
                                                                 OEMEbsdScanSelectionParameter::EbsdReaderType::Esprit, OEMEbsdScanSelectionParameter::ExtensionsType{".h5"}));
   params.insert(std::make_unique<Float32Parameter>(k_ZSpacing_Key, "Z Spacing (Microns)", "The spacing in microns between each layer.", 1.0f));
   params.insert(std::make_unique<VectorFloat32Parameter>(k_Origin_Key, "Origin", "The origin of the volume", std::vector<float32>{0.0F, 0.0F, 0.0F}, std::vector<std::string>{"x", "y", "z"}));

src/Plugins/OrientationAnalysis/src/OrientationAnalysis/Filters/ImportH5OimDataFilter.cpp

@@ -64,7 +64,8 @@ Parameters ImportH5OimDataFilter::parameters() const
   // Create the parameter descriptors that are needed for this filter
   params.insertSeparator(Parameters::Separator{"Input Parameters"});
   params.insert(std::make_unique<OEMEbsdScanSelectionParameter>(k_SelectedScanNames_Key, "Scan Names", "The name of the scan in the .h5 file. EDAX can store multiple scans in a single file",
-                                                                OEMEbsdScanSelectionParameter::ValueType{}, OEMEbsdScanSelectionParameter::AllowedManufacturers{EbsdLib::OEM::EDAX},
+                                                                OEMEbsdScanSelectionParameter::ValueType{},
+                                                                /* OEMEbsdScanSelectionParameter::AllowedManufacturers{EbsdLib::OEM::EDAX},*/
                                                                 OEMEbsdScanSelectionParameter::EbsdReaderType::Oim, OEMEbsdScanSelectionParameter::ExtensionsType{".h5"}));
   params.insert(std::make_unique<Float32Parameter>(k_ZSpacing_Key, "Z Spacing (Microns)", "The spacing in microns between each layer.", 1.0f));
   params.insert(std::make_unique<VectorFloat32Parameter>(k_Origin_Key, "Origin", "The origin of the volume", std::vector<float32>{0.0F, 0.0F, 0.0F}, std::vector<std::string>{"x", "y", "z"}));