Skip to content

Commit

Permalink
Rename ImportCSVData to ImportTextData.
Browse files Browse the repository at this point in the history
Signed-off-by: Joey Kleingers <[email protected]>
  • Loading branch information
joeykleingers authored and imikejackson committed Oct 9, 2023
1 parent a97c9da commit 4a0d178
Show file tree
Hide file tree
Showing 34 changed files with 353 additions and 346 deletions.
6 changes: 3 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ set(CoreParameters
MultiPathSelectionParameter
NumberParameter
NumericTypeParameter
ImportCSVDataParameter
ImportTextDataParameter
ImportHDF5DatasetParameter
StringParameter
VectorParameter
Expand Down Expand Up @@ -431,7 +431,7 @@ set(COMPLEX_HDRS
${COMPLEX_SOURCE_DIR}/Filter/Actions/CreateGridMontageAction.hpp

${COMPLEX_SOURCE_DIR}/Parameters/util/DynamicTableInfo.hpp
${COMPLEX_SOURCE_DIR}/Parameters/util/CSVImporterData.hpp
${COMPLEX_SOURCE_DIR}/Parameters/util/TextImporterData.hpp

${COMPLEX_SOURCE_DIR}/Pipeline/AbstractPipelineNode.hpp
${COMPLEX_SOURCE_DIR}/Pipeline/Pipeline.hpp
Expand Down Expand Up @@ -630,7 +630,7 @@ set(COMPLEX_SRCS
${COMPLEX_SOURCE_DIR}/Filter/Actions/CreateAttributeMatrixAction.cpp
${COMPLEX_SOURCE_DIR}/Filter/Actions/CreateGridMontageAction.cpp

${COMPLEX_SOURCE_DIR}/Parameters/util/CSVImporterData.cpp
${COMPLEX_SOURCE_DIR}/Parameters/util/TextImporterData.cpp
${COMPLEX_SOURCE_DIR}/Parameters/util/DynamicTableInfo.cpp

${COMPLEX_SOURCE_DIR}/Pipeline/AbstractPipelineNode.cpp
Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/ComplexCore/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ set(FilterList
ImageContouringFilter
IdentifySample
ImportBinaryCTNorthstarFilter
ImportCSVDataFilter
ImportTextDataFilter
ImportDeformKeyFileV12Filter
ImportDREAM3DFilter
ImportHDF5Dataset
Expand Down
Original file line number Diff line number Diff line change
@@ -1,64 +1,64 @@
# Import CSV Data
# Import Text Data

## Group (Subgroup) ##

IO (Input)

## Description ##

This **Filter** reads CSV data from any text-based file and imports the data into DREAM3D-NX-style arrays. The user specifies which file to import, how the data is formatted, what to call each array, and what type each array should be.
This **Filter** reads text data from any text-based file and imports the data into DREAM3D-NX-style arrays. The user specifies which file to import, how the data is formatted, what to call each array, and what type each array should be.

*Note:* This **Filter** is intended to read data that is column-oriented, such that each created DREAM3D-NX array corresponds to a column of data in the CSV file. Therefore, this **Filter** will only import scalar arrays. If multiple columns are in fact different components of the same array, then the columns may be imported as separate arrays and then combined in the correct order using the Combine Attribute Arrays **Filter**.
*Note:* This **Filter** is intended to read data that is column-oriented, such that each created DREAM3D-NX array corresponds to a column of data in the text file. Therefore, this **Filter** will only import scalar arrays. If multiple columns are in fact different components of the same array, then the columns may be imported as separate arrays and then combined in the correct order using the Combine Attribute Arrays **Filter**.

### Filling Out The Inputs ###

The user first selects the **Input CSV File** path, which then enables the rest of the interface.
The user first selects the **Input Text File** path, which then enables the rest of the interface.

![Input CSV File Field](Images/Import_CSV_1.png)
![Input Text File Field](Images/Import_Text_1.png)

If the chosen **Input CSV File** already has headers inside the file, the user can select the **Input File Has Headers** checkbox. This
If the chosen **Input Text File** already has headers inside the file, the user can select the **Input File Has Headers** checkbox. This
enables the **Headers Line Number** spin box where the user can select which line of the file contains the headers.

*NOTE*: The interface only allows importing data starting at the line after the chosen **Headers Line Number**. So, in the example below, the **Headers Line Number** is set to 1, so **Start Import Line Number** defaults to 2 and has a range of 2-297 (this particular input file has 297 total lines). The max range of **Headers Line Number** is, of course, set to 296 so that at least 1 line can be imported.

![Input CSV File Field](Images/Import_CSV_2.png)
![Input Text File Field](Images/Import_Text_2.png)

The user can choose how the data is delimited: comma (,), tab, semicolon (;) or space ( ). The user may also elect to ignore consecutive delimiters, which treats consecutive delimiters as one delimiter.

![Input CSV File Field](Images/Import_CSV_3.png)
![Input Text File Field](Images/Import_Text_3.png)

The user can select the number of preview lines available by changing the **Number of Preview Lines** spin box. The range in the example is set to 1-296 because the import is currently starting at row 2 (from **Start Import Line Number** spin box).

![Input CSV File Field](Images/Import_CSV_4.png)
![Input Text File Field](Images/Import_Text_4.png)

The user can then set the data format for each column. Selecting one or more columns will enable the **Column Data Type** combo box, where you can choose a data type or decide to skip importing specific columns as well.

![Input CSV File Field](Images/Import_CSV_5.png)
![Input CSV File Field](Images/Import_CSV_6.png)
![Input Text File Field](Images/Import_Text_5.png)
![Input Text File Field](Images/Import_Text_6.png)

If the **Input File Has Headers** checkbox is OFF, then it is also possible to double-click the headers in the Preview Table to edit them. These values will be used as the name of the **Data Array** in DREAM3D-NX.

*NOTE:* Editing table headers is only available when the **Input File Has Headers** checkbox is OFF. If the **Input File Has Headers** checkbox is ON, then the headers will be read from the **Headers Line Number** in the data file, and the table headers will not be editable.

![Input CSV File Field](Images/Import_CSV_7.png)
![Input Text File Field](Images/Import_Text_7.png)

The user can select the tuple dimensions that will be applied to the imported arrays.

![Input CSV File Field](Images/Import_CSV_8.png)
![Input Text File Field](Images/Import_Text_8.png)

The imported arrays can be stored in either an existing attribute matrix or a new attribute matrix can be created.

![Input CSV File Field](Images/Import_CSV_9.png)
![Input Text File Field](Images/Import_Text_9.png)

Afterwards, you end up with a data structure that looks like this:

![Input CSV File Field](Images/Import_CSV_10.png)
![Input Text File Field](Images/Import_Text_10.png)

## Parameters ##

| Name | Type | Description |
|----------------------------------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Importer Data Object | CSVImporterData | The object that holds all data relevant to importing the data, such as input file path, custom headers, start import line number, data types for all the imported arrays, headers line number, header mode, imported array tuple dimensions, delimiters, etc. |
| Importer Data Object | TextImporterData | The object that holds all data relevant to importing the data, such as input file path, custom headers, start import line number, data types for all the imported arrays, headers line number, header mode, imported array tuple dimensions, delimiters, etc. |
| Use Existing Attribute Matrix | bool | Determines whether or not to store the imported data arrays in an existing attribute matrix |
| Existing Attribute Matrix (Use Existing Attribute Matrix - ON) | DataPath | The data path to the existing attribute matrix where the imported arrays will be stored |
| New Attribute Matrix (Use Existing Attribute Matrix - OFF) | DataPath | The data path to the newly created attribute matrix where the imported arrays will be stored |
Expand All @@ -75,9 +75,9 @@ Not Applicable

## Created Objects ##

| Kind | Default Name | Type | Component Dimensions | Description |
|------|--------------|------|----------------------|-------------|
| One or more **Element/Feature/Ensemble/etc. Data Arrays** | None | Any | 1 | One or more arrays that are created due to importing CSV data |
| Kind | Default Name | Type | Component Dimensions | Description |
|------|--------------|------|----------------------|----------------------------------------------------------------|
| One or more **Element/Feature/Ensemble/etc. Data Arrays** | None | Any | 1 | One or more arrays that are created due to importing text data |

## Example Pipelines ##

Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/ComplexCore/pipelines/Import_ASCII.d3dpipeline
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@
},
"comments": "",
"filter": {
"name": "complex::ImportCSVDataFilter",
"name": "complex::ImportTextDataFilter",
"uuid": "373be1f8-31cf-49f6-aa5d-e356f4f3f261"
},
"isDisabled": false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
},
"comments": "",
"filter": {
"name": "complex::ImportCSVDataFilter",
"name": "complex::ImportTextDataFilter",
"uuid": "373be1f8-31cf-49f6-aa5d-e356f4f3f261"
},
"isDisabled": false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
#include "ComplexCore/Filters/GenerateColorTableFilter.hpp"
#include "ComplexCore/Filters/IdentifySample.hpp"
#include "ComplexCore/Filters/ImportBinaryCTNorthstarFilter.hpp"
#include "ComplexCore/Filters/ImportCSVDataFilter.hpp"
#include "ComplexCore/Filters/ImportTextDataFilter.hpp"
#include "ComplexCore/Filters/ImportDeformKeyFileV12Filter.hpp"
#include "ComplexCore/Filters/ImportDREAM3DFilter.hpp"
#include "ComplexCore/Filters/ImportHDF5Dataset.hpp"
Expand Down Expand Up @@ -166,7 +166,7 @@ namespace complex
{complex::Uuid::FromString("0d0a6535-6565-51c5-a3fc-fbc00008606d").value(), complex::FilterTraits<GenerateColorTableFilter>::uuid}, // GenerateColorTable
{complex::Uuid::FromString("0e8c0818-a3fb-57d4-a5c8-7cb8ae54a40a").value(), complex::FilterTraits<IdentifySample>::uuid}, // IdentifySample
{complex::Uuid::FromString("f2259481-5011-5f22-9fcb-c92fb6f8be10").value(), complex::FilterTraits<ImportBinaryCTNorthstarFilter>::uuid}, // ImportBinaryCTNorthstarFilter
{complex::Uuid::FromString("bdb978bc-96bf-5498-972c-b509c38b8d50").value(), complex::FilterTraits<ImportCSVDataFilter>::uuid}, // ReadASCIIData
{complex::Uuid::FromString("bdb978bc-96bf-5498-972c-b509c38b8d50").value(), complex::FilterTraits<ImportTextDataFilter>::uuid}, // ReadASCIIData
{complex::Uuid::FromString("043cbde5-3878-5718-958f-ae75714df0df").value(), complex::FilterTraits<ImportDREAM3DFilter>::uuid}, // DataContainerReader
{complex::Uuid::FromString("9e98c3b0-5707-5a3b-b8b5-23ef83b02896").value(), complex::FilterTraits<ImportHDF5Dataset>::uuid}, // ImportHDF5Dataset
{complex::Uuid::FromString("a7007472-29e5-5d0a-89a6-1aed11b603f8").value(), complex::FilterTraits<ImportTextFilter>::uuid}, // ImportAsciDataArray
Expand Down
Loading

0 comments on commit 4a0d178

Please sign in to comment.