ENH: Combine and Write STL Files - Allow user to select a Part Number…

… Index to use for numbering. (#1024)

## Combine STL Files

- ENH: Combine STL Files messaging. 
- Change File Index type to Int32
- Store the File List as Feature Data. The Part Number array and the File List are linked via the Part Number Value.

## Write STL Files

-Write files by Part Number
- Allow user to select an Int32 Array to number the output STL files. This is a single
component array at the Triangle/Face level. This will match up with the Combine STL files
filter that generates the Part Number Array
- Parallelize the Part Number writing section.
- AtomicFile: Use unique error codes and better error message formatting.
- Add unit test for Part Number code path for STL Writer

## Misc Changes

- Move MD5 class into simplnx/Utilities since it can be used elsewhere in the code

Signed-off-by: Michael Jackson <[email protected]>

CMakeLists.txt

@@ -548,6 +548,7 @@ set(SIMPLNX_HDRS
   ${SIMPLNX_SOURCE_DIR}/Utilities/Parsing/HDF5/Writers/ObjectWriter.hpp
 
   ${SIMPLNX_SOURCE_DIR}/Utilities/Parsing/Text/CsvParser.hpp
+  ${SIMPLNX_SOURCE_DIR}/Utilities/MD5.hpp
 )
 
 set(SIMPLNX_GENERATED_HEADERS
@@ -743,6 +744,7 @@ set(SIMPLNX_SRCS
   ${SIMPLNX_SOURCE_DIR}/Utilities/Parsing/HDF5/Writers/ObjectWriter.cpp
 
   ${SIMPLNX_SOURCE_DIR}/Utilities/Parsing/Text/CsvParser.cpp
+  ${SIMPLNX_SOURCE_DIR}/Utilities/MD5.cpp
 )
 
 # Add Core FilterParameters

src/Plugins/ITKImageProcessing/test/CMakeLists.txt

@@ -98,8 +98,6 @@ target_sources(${PLUGIN_NAME}UnitTest
   PRIVATE
     ${${PLUGIN_NAME}_SOURCE_DIR}/test/ITKTestBase.hpp
     ${${PLUGIN_NAME}_SOURCE_DIR}/test/ITKTestBase.cpp
-    ${${PLUGIN_NAME}_SOURCE_DIR}/test/MD5.hpp
-    ${${PLUGIN_NAME}_SOURCE_DIR}/test/MD5.cpp
 )
 
 # -----------------------------------------------------------------------------

src/Plugins/ITKImageProcessing/test/ITKTestBase.cpp

@@ -2,7 +2,6 @@
 
 #include "ITKImageProcessing/Filters/ITKImageReaderFilter.hpp"
 #include "ITKImageProcessing/Filters/ITKImageWriterFilter.hpp"
-#include "MD5.hpp"
 
 #include <itkImportImageFilter.h>
 #include <itkNumericTraits.h>
@@ -12,6 +11,7 @@
 #include <itkTestingHashImageFilter.h>
 
 #include "simplnx/Common/Types.hpp"
+#include "simplnx/Utilities/MD5.hpp"
 
 #include <fmt/format.h>
 

src/Plugins/SimplnxCore/docs/CombineStlFilesFilter.md

@@ -2,13 +2,16 @@
 
 ## Group (Subgroup)
 
-Reader/Input (AMProcessMonitoring)
+Reader/Input
 
 ## Description
 
 This **Filter** combines all of the STL files from a given directory into a single triangle geometry. This filter will make use of the **Import STL File Filter** to read in each stl file in the given directory and then will proceed to combine each of the imported files into a single triangle geometry.
 
-There is an option to label the triangles and vertices with the "index" of the file that was read. This would be based on the lexographical index and starts from 1. This allows for the immediate "segmentation" of the resulting triangle geometry or just as a convenience to "color by" in the visualization widget.
+There is an option to label the faces and vertices with a "Part Number" that represents the index into the list of files that was used as the input. This would be based on the lexographical index and starts from 1. This allows for the immediate "segmentation" of the resulting triangle geometry or just as a convenience to "color by" in the visualization widget. This can
+also be used in the "Write STL Files from Triangle Geometry" Filter if the selection for
+the "File Grouping Type" is set to "Part Index" in the UI. Then use the "Part Number" array
+that is created in this filter for the "Part Index". 
 
 % Auto generated parameter table will be inserted here
 

src/Plugins/SimplnxCore/docs/WriteStlFileFilter.md

@@ -6,7 +6,20 @@ IO (Output)
 
 ## Description
 
-This **Filter** will write a binary STL File for each unique **Feature** Id in the associated **Triangle** geometry. The STL files will be named with the [Feature_Id].stl. The user can designate an optional prefix for the files.
+This **Filter** can generate a single or multiple binary STL (StereoLithography) 
+files for a given Triangle Geometry. This is controlled by the "File Grouping Type"
+parameter which is a combo box parameter type with the following choices:
+
+- Features: The user must supply a 2 component Int32 array where the values are the 2 features that the triangle belongs to.
+- Phases and Features: The user must the same kind of array as in the "Features" but also
+ must supply a single component array that represents another higher order of grouping, such
+ as a phase (in the case of microstructure data) or a part number in the base of importing
+ a large multipart geometry.
+- Single File: The entire Triangle Geometry is saved as a single STL File
+- Part Index: The user must supply a single component Int32 array at the triangle 
+ level that represents which part the triangle belongs to. This is handy when the Triangle Geometry
+ has distinct, non-overlapping "parts", such as an AM build with.
+
 
 % Auto generated parameter table will be inserted here
 

src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CombineStlFiles.cpp

@@ -82,25 +82,52 @@ const std::atomic_bool& CombineStlFiles::getCancel()
 Result<> CombineStlFiles::operator()()
 {
   DataStructure tempDataStructure;
-  for(const auto& dirEntry : std::filesystem::directory_iterator{m_InputValues->StlFilesPath})
+  std::vector<fs::path> paths;
+  const std::string ext(".stl");
+
+  // Just count up the stl files in the directory
+  size_t index = 0;
+  for(const auto& entry : std::filesystem::directory_iterator{m_InputValues->StlFilesPath})
+  {
+    if(fs::is_regular_file(entry) && StringUtilities::toLower(entry.path().extension().string()) == ext)
+    {
+      paths.emplace_back(entry);
+    }
+  }
+
+  // Sort the paths for something sort of reasonable.
+  std::sort(paths.begin(), paths.end());
+
+  auto pCellFeatureAttributeMatrixPath = m_InputValues->TriangleDataContainerName.createChildPath(m_InputValues->CellFeatureAttributeMatrixName);
+  auto activeArrayPath = pCellFeatureAttributeMatrixPath.createChildPath(m_InputValues->ActiveArrayName);
+  auto fileListPath = pCellFeatureAttributeMatrixPath.createChildPath(m_InputValues->FileListArrayName);
+
+  auto activeArray = m_DataStructure.getDataRefAs<UInt8Array>(activeArrayPath);
+  activeArray[0] = 0;
+  auto fileListStrArray = m_DataStructure.getDataRefAs<StringArray>(fileListPath);
+
+  int32 currentIndex = 1;
+  for(const auto& filePath : paths)
   {
+    std::string stlFilePath = filePath.string();
     if(getCancel())
     {
       return {};
     }
 
-    const fs::path& stlFilePath = dirEntry.path();
-    if(fs::is_regular_file(stlFilePath) && StringUtilities::toLower(stlFilePath.extension().string()) == ".stl")
+    fileListStrArray[currentIndex] = stlFilePath;
+    activeArray[currentIndex] = 1;
+    m_MessageHandler(IFilter::Message::Type::Info, fmt::format("({}/{}) Reading {}", currentIndex, paths.size(), stlFilePath));
+    currentIndex++;
+
+    ReadStlFileFilter stlFileReader;
+    Arguments args;
+    args.insertOrAssign(ReadStlFileFilter::k_StlFilePath_Key, std::make_any<FileSystemPathParameter::ValueType>(stlFilePath));
+    args.insertOrAssign(ReadStlFileFilter::k_CreatedTriangleGeometryPath_Key, std::make_any<DataPath>(DataPath({filePath.stem().string()})));
+    auto executeResult = stlFileReader.execute(tempDataStructure, args);
+    if(executeResult.result.invalid())
     {
-      ReadStlFileFilter stlFileReader;
-      Arguments args;
-      args.insertOrAssign(ReadStlFileFilter::k_StlFilePath_Key, std::make_any<FileSystemPathParameter::ValueType>(stlFilePath));
-      args.insertOrAssign(ReadStlFileFilter::k_CreatedTriangleGeometryPath_Key, std::make_any<DataPath>(DataPath({stlFilePath.stem().string()})));
-      auto executeResult = stlFileReader.execute(tempDataStructure, args);
-      if(executeResult.result.invalid())
-      {
-        return executeResult.result;
-      }
+      return executeResult.result;
     }
   }
 
@@ -144,6 +171,8 @@ Result<> CombineStlFiles::operator()()
   usize faceLabelOffset = 0;
   usize vertexLabelOffset = 0;
 
+  m_MessageHandler(IFilter::Message::Type::Info, fmt::format("Moving final triangle geometry data..."));
+
   // Loop over each temp geometry and copy the data into the destination geometry
   for(auto* currentGeometry : stlGeometries)
   {
@@ -167,23 +196,15 @@ Result<> CombineStlFiles::operator()()
 
     if(m_InputValues->LabelFaces)
     {
-      auto& faceLabels = m_DataStructure.getDataRefAs<UInt32Array>(m_InputValues->FaceFileIndexArrayPath);
-      //      for(usize tuple = faceLabelOffset; tuple < faceLabelOffset + currentGeomNumTriangles; tuple++)
-      //      {
-      //        faceLabels[tuple] = fileIndex;
-      //      }
+      auto& faceLabels = m_DataStructure.getDataRefAs<Int32Array>(m_InputValues->FaceFileIndexArrayPath);
       std::fill(faceLabels.begin() + faceLabelOffset, faceLabels.begin() + faceLabelOffset + currentGeomNumTriangles, fileIndex);
     }
 
     faceLabelOffset += currentGeomNumTriangles;
 
     if(m_InputValues->LabelVertices)
     {
-      auto& vertexLabels = m_DataStructure.getDataRefAs<UInt32Array>(m_InputValues->VertexFileIndexArrayPath);
-      //      for(usize tuple = vertexLabelOffset; tuple < vertexLabelOffset + currentGeomNumVertices; tuple++)
-      //      {
-      //        vertexLabels[tuple] = fileIndex;
-      //      }
+      auto& vertexLabels = m_DataStructure.getDataRefAs<Int32Array>(m_InputValues->VertexFileIndexArrayPath);
       std::fill(vertexLabels.begin() + vertexLabelOffset, vertexLabels.begin() + vertexLabelOffset + currentGeomNumVertices, fileIndex);
     }
     vertexLabelOffset += currentGeomNumVertices;

src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/CombineStlFiles.hpp

@@ -5,6 +5,7 @@
 #include "simplnx/DataStructure/DataPath.hpp"
 #include "simplnx/DataStructure/DataStructure.hpp"
 #include "simplnx/Filter/IFilter.hpp"
+#include "simplnx/Parameters/DataObjectNameParameter.hpp"
 #include "simplnx/Parameters/FileSystemPathParameter.hpp"
 #include "simplnx/Parameters/StringParameter.hpp"
 
@@ -21,6 +22,9 @@ struct SIMPLNXCORE_EXPORT CombineStlFilesInputValues
   bool LabelFaces;
   DataPath VertexFileIndexArrayPath;
   bool LabelVertices;
+  DataObjectNameParameter::ValueType CellFeatureAttributeMatrixName;
+  DataObjectNameParameter::ValueType ActiveArrayName;
+  DataObjectNameParameter::ValueType FileListArrayName;
 };
 
 /**