Skip to content

Commit

Permalink
PY: Enable testing python example codes (#806)
Browse files Browse the repository at this point in the history
Signed-off-by: Michael Jackson <[email protected]>
  • Loading branch information
imikejackson authored Dec 27, 2023
1 parent 0e9ce25 commit cded997
Show file tree
Hide file tree
Showing 36 changed files with 278 additions and 270 deletions.
12 changes: 6 additions & 6 deletions .azure/azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
timeoutInMinutes: 120

variables:
complex_source_dir: $(Build.Repository.LocalPath)
simplnx_source_dir: $(Build.Repository.LocalPath)
build_type: Release
dream3d_data_dir: $(Agent.WorkFolder)/DREAM3D_Data

Expand All @@ -58,25 +58,25 @@ jobs:
displayName: Appending to PATH
- script: |
cmake -S $(complex_source_dir) -B $(Build.BinariesDirectory) -G Ninja -DCMAKE_BUILD_TYPE:STRING=$(build_type) -DDREAM3D_DATA_DIR=$(dream3d_data_dir) -C $(Agent.WorkFolder)/NX.cmake -DBUILDNAME:STRING="simplnx-$(Agent.MachineName)-$(Agent.OS)-PR$(System.PullRequest.PullRequestNumber)"
cmake -S $(simplnx_source_dir) -B $(Build.BinariesDirectory) -G Ninja -DCMAKE_BUILD_TYPE:STRING=$(build_type) -DDREAM3D_DATA_DIR=$(dream3d_data_dir) -C $(Agent.WorkFolder)/NX.cmake -DBUILDNAME:STRING="simplnx-$(Agent.MachineName)-$(Agent.OS)-PR$(System.PullRequest.PullRequestNumber)"
displayName: Configure COMPLEX
displayName: Configure SIMPLNX
- script: |
cmake --build $(Build.BinariesDirectory) --config $(build_type) --target all
displayName: Build COMPLEX
displayName: Build SIMPLNX
- script: |
cd $(Build.BinariesDirectory)
cpack -C $(build_type) --verbose
continueOnError: true
displayName: Packing COMPLEX
displayName: Packing SIMPLNX
- script: |
cd $(Build.BinariesDirectory)
ctest -C $(build_type) -D Experimental --timeout 7200 -DCTEST_SITE:STRING=$(Agent.MachineName).bluequartz.net -Ddashboard_source_name:STRING=DREAM3D
continueOnError: true
displayName: Testing COMPLEX
displayName: Testing SIMPLNX
- task: PublishTestResults@2
inputs:
Expand Down
6 changes: 5 additions & 1 deletion CMakePresets.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,11 @@
"type": "BOOL",
"value": "ON"
},
"SIMPLNX_ENABLE_PYTHON_TESTS": {
"SIMPLNX_BUILD_PYTHON_TESTS": {
"type": "BOOL",
"value": "ON"
},
"SIMPLNX_BUILD_PYTHON_DOCS": {
"type": "BOOL",
"value": "ON"
},
Expand Down
6 changes: 3 additions & 3 deletions PortingFilters.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ follows:
This custom build has **ALL** ***SIMPL*** plugins compiled so you don't need
to worry about what filters will be available

### Load up ***SIMPL*** DREAM3D and navigate to ***ComplexFilterGen***
### Load up ***SIMPL*** DREAM3D and navigate to ***SimplnxFilterGen***

Here you will need to set the command arguments using the following syntax:

Expand Down Expand Up @@ -120,9 +120,9 @@ The syntax for use of ***filterList*** is as follows:

## SECTION 4 : Multithreading

At the current time, the only filters that should be made parallel are those that could be considered "embarrassingly parallel". It is important to remember that the cost of creating a thread is hefty so it should only be done when there is a sizeable amount of work available for each thread. Complex has two types: ParallelTaskAlgorithm and ParallelDataAlgorithm. Task Runner is for parsing multiple objects and Data Runner is for parsing a single object.
At the current time, the only filters that should be made parallel are those that could be considered "embarrassingly parallel". It is important to remember that the cost of creating a thread is hefty so it should only be done when there is a sizeable amount of work available for each thread. Simplnx has two types: ParallelTaskAlgorithm and ParallelDataAlgorithm. Task Runner is for parsing multiple objects and Data Runner is for parsing a single object.

### Syntax for Complex
### Syntax for Simplnx

This is an examplar use case and doesn't truly encompass all possible use cases for the functions, but instead serves to show how it should be structured in most cases.

Expand Down
2 changes: 1 addition & 1 deletion SUPPORT.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Complex Support #
# Simplnx Support #

## Documentation ##

Expand Down
2 changes: 1 addition & 1 deletion cmake/SIMPLNXVersion.cpp.in
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*--------------------------------------------------------------------------
* This file is autogenerated from:
* @CMAKE_CURRENT_LIST_DIR@/ComplexVersion.cpp.in
* @CMAKE_CURRENT_LIST_DIR@/SIMPLNXVersion.cpp.in
* during the cmake configuration of your project. If you need to make changes,
* edit the original file NOT THIS FILE.
* --------------------------------------------------------------------------*/
Expand Down
11 changes: 6 additions & 5 deletions cmake/Summary.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ foreach(proj ${ALL_PROJECTS})
message(STATUS "* ${proj}: ${${proj}_SOURCE_DIR} Git Hash: ${githash}")
endforeach()
message(STATUS "* BUILD_TYPE: ${CMAKE_BUILD_TYPE}")
message(STATUS "* COMPLEX: ${simplnx_VERSION}")
message(STATUS "* SIMPLNX: ${simplnx_VERSION}")
OSInformation()
message(STATUS "* -------------- Dependent Libraries -------------------------------------------")
message(STATUS "* Eigen (${Eigen3_VERSION}) ${Eigen3_DIR}")
Expand All @@ -60,7 +60,7 @@ message(STATUS "* boost_mp11 (${boost_mp11_VERSION}) ${boost_mp11_DIR}")
message(STATUS "* nod (${nod_VERSION}) ${nod_DIR}")
message(STATUS "* reproc++ (${reproc_VERSION}) ${reproc++_DIR}")

message(STATUS "* -------------- Complex Configuration Options -------------------------------------")
message(STATUS "* -------------- Simplnx Configuration Options -------------------------------------")
message(STATUS "* SIMPLNX_BUILD_TESTS: ${SIMPLNX_BUILD_TESTS}")
message(STATUS "* SIMPLNX_ENABLE_MULTICORE: ${SIMPLNX_ENABLE_MULTICORE}")
message(STATUS "* SIMPLNX_ENABLE_COMPRESSORS: ${SIMPLNX_ENABLE_COMPRESSORS}")
Expand All @@ -72,12 +72,13 @@ message(STATUS "* SIMPLNX_ENABLE_PACKAGING: ${SIMPLNX_ENABLE_PACKAGING}")
message(STATUS "* SIMPLNX_BUILD_DOCS: ${SIMPLNX_BUILD_DOCS}")
message(STATUS "* DREAM3D_DATA_DIR: ${DREAM3D_DATA_DIR}")

message(STATUS "* -------------- Complex Python Configuration Options -------------------------------------")
message(STATUS "* -------------- Simplnx Python Configuration Options -------------------------------------")
message(STATUS "* Python3_EXECUTABLE: ${Python3_EXECUTABLE}")
message(STATUS "* SIMPLNX_BUILD_PYTHON: ${SIMPLNX_BUILD_PYTHON}")
message(STATUS "* SIMPLNX_EMBED_PYTHON: ${SIMPLNX_EMBED_PYTHON}")
message(STATUS "* SIMPLNX_ENABLE_SPHINX_DOCS: ${SIMPLNX_ENABLE_SPHINX_DOCS}")
message(STATUS "* SIMPLNX_BUILD_PYTHON_DOCS: ${SIMPLNX_BUILD_PYTHON_DOCS}")
message(STATUS "* SIMPLNX_BUILD_PYTHON_TESTS: ${SIMPLNX_BUILD_PYTHON_TESTS}")
message(STATUS "* SIMPLNX_CONDA_BUILD: ${SIMPLNX_CONDA_BUILD}")
message(STATUS "* SIMPLNX_EMBED_PYTHON: ${SIMPLNX_EMBED_PYTHON}")

# message(STATUS "* -------------- Known Plugins ------------------------------------------------------")
# foreach(plugin ${SIMPLNX_PLUGIN_LIST})
Expand Down
8 changes: 4 additions & 4 deletions cmake/Utility.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -474,10 +474,10 @@ function(AddPythonTest)
set(multiValueArgs PYTHONPATH)
cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
message(STATUS "ARGS_FILE:${ARGS_FILE}")
if(COMPLEX_BUILD_PYTHON)
if(SIMPLNX_BUILD_PYTHON)
if(WIN32)
add_test(NAME ${ARGS_NAME}
COMMAND ${complex_SOURCE_DIR}/wrapping/python/testing/anaconda_test.bat
COMMAND ${simplnx_SOURCE_DIR}/wrapping/python/testing/anaconda_test.bat
)

set_property(TEST ${ARGS_NAME}
Expand All @@ -488,7 +488,7 @@ function(AddPythonTest)
)
else()
add_test(NAME ${ARGS_NAME}
COMMAND ${complex_SOURCE_DIR}/wrapping/python/testing/anaconda_test.sh
COMMAND ${simplnx_SOURCE_DIR}/wrapping/python/testing/anaconda_test.sh
)
set_property(TEST ${ARGS_NAME}
PROPERTY
Expand Down Expand Up @@ -530,7 +530,7 @@ function(CreatePythonTests)
cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})

set(TESTS_PYTHONPATH
"$<TARGET_FILE_DIR:complex>"
"$<TARGET_FILE_DIR:simplnx>"
)

foreach(test ${ARGS_TEST_NAMES})
Expand Down
2 changes: 1 addition & 1 deletion docs/Build_From_Source.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ Be default VCPKG will check for updates to the libraries that it compiles. If yo

## Defining where test data is stored ##

Complex and its plugins require test files to be able to perform the unit tests. By default these will be store inside the build directory. This means that if you have multiple build directories, a separate copy of all the test files will be downloaded for each build directory. The developer can set the `DREAM3D_DATA_DIR` variable to a path that will be used. They will need to set this for **each** build directory. You **MUST** define `DREAM3D_DATA_DIR` using an absolute path. Relative paths **will not work**.
Simplnx and its plugins require test files to be able to perform the unit tests. By default these will be store inside the build directory. This means that if you have multiple build directories, a separate copy of all the test files will be downloaded for each build directory. The developer can set the `DREAM3D_DATA_DIR` variable to a path that will be used. They will need to set this for **each** build directory. You **MUST** define `DREAM3D_DATA_DIR` using an absolute path. Relative paths **will not work**.

+ `-DDREAM3D_DATA_DIR=/opt/local/DREAM3D_Data`

Expand Down
4 changes: 2 additions & 2 deletions docs/Code_Style_Guide.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Code Style Guide

This is the style guide for the COMPLEX library. When contributing sources to the simplnx repository we ask that all contributions follow this style guide. These general rules have been developed over the years in an effort to allow efficient coding practices.
This is the style guide for the SIMPLNX library. When contributing sources to the simplnx repository we ask that all contributions follow this style guide. These general rules have been developed over the years in an effort to allow efficient coding practices.

## Source Code Line Endings

Expand Down Expand Up @@ -72,7 +72,7 @@ Note that with C++11, programmers now have the ability to inform the compiler wh

## Naming conventions

There is an include clang-tidy file that most IDE's can use to assist with some of the major areas. One area is the naming conventions that COMPLEX uses. Here are the basic rules:
There is an include clang-tidy file that most IDE's can use to assist with some of the major areas. One area is the naming conventions that SIMPLNX uses. Here are the basic rules:

- Class and Structs are `UpperCamelCase`
- Class private member variables are `m_UpperCamelCase`
Expand Down
4 changes: 2 additions & 2 deletions docs/Porting_Filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@ static inline constexpr StringLiteral k_AlignmentType_Key = "alignment_type";
- `QMap<> => std::map<>`
- `QByteArray => std::array<int8> or std::vector<int8>`

## Converting `setErrorCondition` from SIMPL to COMPLEX ##
## Converting `setErrorCondition` from SIMPL to SIMPLNX ##

SIMPL

```cpp
setErrorCondition(nx::core::StlConstants::k_ErrorOpeningFile, "Error opening STL file");
```
COMPLEX
SIMPLNX
```cpp
Result<> result = MakeErrorResult(nx::core::StlConstants::k_ErrorOpeningFile, "Error opening STL file")
Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/OrientationAnalysis/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ endforeach()
target_include_directories(simplnx PUBLIC )

# ------------------------------------------------------------------------------
# Add a link library to simplnx because it now depends on EbsdLib. Complex will
# Add a link library to simplnx because it now depends on EbsdLib. Simplnx will
# be able to find the include dirs from the CMake EbsdLib target
# ------------------------------------------------------------------------------
target_link_libraries(simplnx PUBLIC EbsdLib::EbsdLib)
Expand Down
6 changes: 3 additions & 3 deletions src/Plugins/SimplnxCore/test/MoveDataTest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ DataStructure createDataStructure()
}
} // namespace

TEST_CASE("SimplnxCore::MoveData Successful", "[Complex::Core][MoveData]")
TEST_CASE("SimplnxCore::MoveData Successful", "[Simplnx::Core][MoveData]")
{
MoveData filter;
Arguments args;
Expand Down Expand Up @@ -79,7 +79,7 @@ TEST_CASE("SimplnxCore::MoveData Successful", "[Complex::Core][MoveData]")
REQUIRE(dataStructure.getDataAs<DataGroup>(newGroup4Path) != nullptr);
}

TEST_CASE("SimplnxCore::MoveData Unsuccessful", "[Complex::Core][MoveData]")
TEST_CASE("SimplnxCore::MoveData Unsuccessful", "[Simplnx::Core][MoveData]")
{
MoveData filter;
Arguments args;
Expand Down Expand Up @@ -117,7 +117,7 @@ TEST_CASE("SimplnxCore::MoveData Unsuccessful", "[Complex::Core][MoveData]")
}
}

TEST_CASE("SimplnxCore::MoveData Tuple Size Mismatches Warning and Failure", "[Complex::Core][MoveData]")
TEST_CASE("SimplnxCore::MoveData Tuple Size Mismatches Warning and Failure", "[Simplnx::Core][MoveData]")
{
MoveData filter;
Arguments args;
Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/SimplnxCore/test/PipelineTest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ TEST_CASE("PipelineTest:Execute Pipeline")
REQUIRE(pipeline.push_back(tf2Handle));
}

TEST_CASE("PipelineTest:Complex Pipeline")
TEST_CASE("PipelineTest:Simplnx Pipeline")
{
auto app = Application::GetOrCreateInstance();
app->loadPlugins(unit_test::k_BuildDir.view());
Expand Down
18 changes: 9 additions & 9 deletions src/simplnx/Core/Application.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -182,13 +182,13 @@ void Application::savePreferences()
m_Preferences->saveToFile(filepath);
}

std::optional<Uuid> Application::getComplexUuid(const Uuid& simplUuid)
std::optional<Uuid> Application::getSimplnxUuid(const Uuid& simplUuid)
{
for(usize index = 0; index < m_Simpl_Uuids.size(); index++)
{
if(m_Simpl_Uuids[index] == simplUuid)
{
return m_Complex_Uuids[index];
return m_Simplnx_Uuids[index];
}
}

Expand All @@ -198,9 +198,9 @@ std::optional<Uuid> Application::getComplexUuid(const Uuid& simplUuid)
std::vector<Uuid> Application::getSimplUuid(const Uuid& simplnxUuid)
{
std::vector<usize> indices;
for(usize index = 0; index < m_Complex_Uuids.size(); index++)
for(usize index = 0; index < m_Simplnx_Uuids.size(); index++)
{
if(m_Complex_Uuids[index] == simplnxUuid)
if(m_Simplnx_Uuids[index] == simplnxUuid)
{
indices.push_back(index);
}
Expand Down Expand Up @@ -294,8 +294,8 @@ void Application::loadPlugin(const std::filesystem::path& path, bool verbose)
return;
}

AbstractPlugin::SIMPLMapType simplToComplexUuids = plugin->getSimplToSimplnxMap();
for(auto const& [simplUuid, simplData] : simplToComplexUuids)
AbstractPlugin::SIMPLMapType simplToSimplnxUuids = plugin->getSimplToSimplnxMap();
for(auto const& [simplUuid, simplData] : simplToSimplnxUuids)
{
for(const auto& uuid : m_Simpl_Uuids)
{
Expand All @@ -305,12 +305,12 @@ void Application::loadPlugin(const std::filesystem::path& path, bool verbose)
}
}
m_Simpl_Uuids.push_back(simplUuid);
m_Complex_Uuids.push_back(simplData.simplnxUuid);
m_Simplnx_Uuids.push_back(simplData.simplnxUuid);
}

if(m_Simpl_Uuids.size() != m_Complex_Uuids.size())
if(m_Simpl_Uuids.size() != m_Simplnx_Uuids.size())
{
throw std::runtime_error(fmt::format("UUID maps are not of the same size! SIMPL UUID Vector size: {} Complex UUID Vector size: {}", m_Simpl_Uuids.size(), m_Complex_Uuids.size()));
throw std::runtime_error(fmt::format("UUID maps are not of the same size! SIMPL UUID Vector size: {} Simplnx UUID Vector size: {}", m_Simpl_Uuids.size(), m_Simplnx_Uuids.size()));
}

for(const auto& pluginIO : plugin->getDataIOManagers())
Expand Down
10 changes: 5 additions & 5 deletions src/simplnx/Core/Application.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -131,13 +131,13 @@ class SIMPLNX_EXPORT Application
std::filesystem::path getCurrentDir() const;

/**
* @brief Returns the Complex filter UUID [v4] from the SIMPL filter UUID [v5]
* @brief Returns the Simplnx filter UUID [v4] from the SIMPL filter UUID [v5]
* @return std::optional<Uuid>
*/
std::optional<Uuid> getComplexUuid(const Uuid& simplUuid);
std::optional<Uuid> getSimplnxUuid(const Uuid& simplUuid);

/**
* @brief Returns the SIMPL filter UUID(s) [v5] from the Complex filter UUID [v4]
* @brief Returns the SIMPL filter UUID(s) [v5] from the Simplnx filter UUID [v4]
* @return std::optional<std::vector<Uuid>>
*/
std::vector<Uuid> getSimplUuid(const Uuid& simplnxUuid);
Expand Down Expand Up @@ -184,8 +184,8 @@ class SIMPLNX_EXPORT Application
// Variables
std::unique_ptr<nx::core::FilterList> m_FilterList;
std::filesystem::path m_CurrentPath = "";
std::vector<Uuid> m_Simpl_Uuids; // no duplicates; index must match m_Complex_Uuids
std::vector<Uuid> m_Complex_Uuids; // duplicate allowed conditionally; index must match m_Simpl_Uuids
std::vector<Uuid> m_Simpl_Uuids; // no duplicates; index must match m_Simplnx_Uuids
std::vector<Uuid> m_Simplnx_Uuids; // duplicate allowed conditionally; index must match m_Simpl_Uuids
std::shared_ptr<DataIOCollection> m_DataIOCollection;
name_type_map m_NamedTypesMap;
std::unique_ptr<Preferences> m_Preferences = nullptr;
Expand Down
2 changes: 1 addition & 1 deletion src/simplnx/DataStructure/IO/Generic/DataIOCollection.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ class SIMPLNX_EXPORT DataIOCollection

/**
* @brief Returns the IDataIOManager for the specified format name.
* Complex comes with HDF5 IO Manager.
* Simplnx comes with HDF5 IO Manager.
* Additional IDataIOManagers are added through plugins.
* @param formatName
* @return std::shared_ptr<IDataIOManager>
Expand Down
7 changes: 5 additions & 2 deletions src/simplnx/DataStructure/ReadMe.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
# simplnx::DataStructure

## Adding DataObject types
When adding DataObject types, additional IDataFactories need to be created for each IDataIOManager subclass. Complex provides the HDF5 version of IDataIOManager and IDataFactory, but plugins providing additional formats will need to be updated to enable reading and writing to the new types.

When adding DataObject types, additional IDataFactories need to be created for each IDataIOManager subclass. Simplnx provides the HDF5 version of IDataIOManager and IDataFactory, but plugins providing additional formats will need to be updated to enable reading and writing to the new types.

## Adding IO formats

simplnx comes with HDF5 readers and writers by default. To add additional IO formats, the plugin developer will require additional classes to tell simplnx how to read and write the DataObjects using the specified format.

Required subclasses:

- IDataFactory for each concrete DataObject type
- IDataIOManager for the format
* Add IDataFactory subclasses to the IO Manager

- Add IDataFactory subclasses to the IO Manager

In addition, a DataStructure reader and writer should be provided for the new format for easy IO.

Expand Down
Loading

0 comments on commit cded997

Please sign in to comment.