Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update data-importer-extension.md #8769

Open
wants to merge 4 commits into
base: development
Choose a base branch
from
Open

Update data-importer-extension.md #8769

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
70f7faf
Update data-importer-extension.md
mahendra-mahajan-mendix Dec 9, 2024
0a59619
High-level review
MariaShaposhnikova Dec 12, 2024
5f2aa59
Proofread
quinntracy Dec 13, 2024
51d0593
Proofread
quinntracy Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 120 additions & 5 deletions ...store/use-content/platform-supported-content/modules/data-importer-extension.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,16 @@ aliases:

## Introduction

{{% alert color="info" %}}
The Data Importer is available in [Studio Pro 10.6](/releasenotes/studio-pro/10.6/) and above.
{{% /alert %}}

The [Data Importer](https://marketplace.mendix.com/link/component/219833) allows you to import data from an Excel or comma-separated value (CSV) file. You can choose which sheet and columns to import, preview the data, and create a non-persistable entity (NPE) in your domain model that corresponds to your input. Then, you can import data into your app using the [Import Data from File](/refguide/import-data-from-file/) activity.

The Data Importer Document can also be used as a source for creating [Import Mapping](/refguide/import-mappings/). This import mapping can import data from Excel/CSV file using the [Import with Mapping](/refguide/import-mapping-action/) activity.

{{% alert color="info" %}}
The Data Importer is available in [Studio Pro 10.6](/releasenotes/studio-pro/10.6/) and above.
The Data Importer document as a source for import mapping is available from [Studio Pro 10.18.0](/releasenotes/studio-pro/10.18/) and above.
{{% /alert %}}

### Typical Use Cases
Expand All @@ -32,7 +38,6 @@ This extension currently has the following limitations:
* Source data can be mapped to one entity only; associations are not currently supported
* You cannot map data to an existing NPE; you have to create a new entity as part of mapping
* Enumerations are not supported
* Multi-line headers and headers containing new-line characters (\r, \n, \r+\n) are not supported
* **String** is the default attribute type (*.csv* only)

### Prerequisites
Expand All @@ -43,7 +48,7 @@ This extension currently has the following limitations:

Download the [Data Importer](https://marketplace.mendix.com/link/component/219833) from the Marketplace and [add it into your app](/appstore/use-content/).

## Data Importer Document
## Data Importer Document (With Implicit Mapping)

### Creating a Data Importer Document {#create-document}

Expand Down Expand Up @@ -124,13 +129,13 @@ Once you are satisfied with the changes, click **OK** to save or **Cancel** to d
The **Edit Entity** feature is useful for CSV import, as all the columns of a CSV file are marked as String by default, so you can change the data type if necessary. The following table shows the source-to-target data conversion matrix:

Input CSV File

| Source Type | Target- String | Target- Int | Target- Long | Target- Decimal | Target- Boolean | Target- DateTime |
| :-------- | :------- | :-------- | :------- | :-------- | :------- | :-------- |
| String | Yes | Partial | Partial | Partial | Partial | No |

Input Excel File

| Source Type | Target- String | Target- Int | Target- Long | Target- Decimal | Target- Boolean | Target- DateTime |
| :-------- | :------- | :-------- | :------- | :-------- | :------- | :-------- |
| String | Yes | Partial | Partial | Partial | Partial | No |
Expand Down Expand Up @@ -188,6 +193,116 @@ To perform testing, you can do the following actions:

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/local-app-run.png" class="no-border" >}}

## Data Importer Document (as a Source for Import Mapping)

The [Import Mapping](/refguide/import-mappings/) document and the [Import with Mapping](/refguide/import-mapping-action/) activity provide inherent advantages, like controlling the commit of objects, and the flexibility to find or create an object. This new feature of the Data Importer document leverages these capabilities by creating a source structure, which can be used to create import mappings. If you are comfortable working with [Mapping Documents](/refguide/mapping-documents/), you can use this feature to address advanced use cases of importing data into Studio Pro. The section below describes how to create a structure, an import mapping using this structure, and leveraging the **Import with Mapping** microflow activity to import data into Studio Pro.

### Creating a Data Importer Document {#create-document-with-imm}

To import data, do the following:

1. Right-click the module and click **Add other** > **Data Importer**.
2. Name the document and enable the **Use with Import Mapping** checkbox.
3. Click **OK** and the new Data Importer document opens.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/create-DI-doc-with-import-mapping.png" class="no-border" >}}

### Previewing Structure {#preview-structure}

Once you have [created the Data Importer document](#create-document-with-imm), click **Select a local file** to upload a sample Excel file (*.xls* or *.xslx*) or CSV file (*.csv*).

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/select-file-for-structure-preview.png" class="no-border" >}}

You can choose which sheet to import data from, and specify the header row and starting data row.

* **Sheet Name** – name of the worksheet from where data needs to be imported; if the file has multiple worksheets, the sheet name appears in the drop-down
* **Header Row No.** – row number of the file header; the default is **1**
* **Read Data From Row No.** – starting line for reading data; the default is **2**

Click **Preview Structure Elements** to view the data structure from the file. The data from the first row of the source file is shown in the **Schema elements** section. Click **Create Structure** and you will be notified that a new structure is generated successfully.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/preview-data-structure.png" class="no-border" >}}

{{% alert color="warning" %}} Column names that do not adhere to Mendix naming conventions will be autocorrected. {{% /alert %}}

### Create Import Mapping {#DI-import-mapping}

Create the import mapping using the Data Importer document created in the step above by doing the following:

1. Right-click your module or folder and select **Add other** > **Import mapping**.
2. Name the import mapping document, then click **OK**. You are then routed to **Select a schema element for import mapping**.
3. From the **Schema source**, choose **Excel/CSV structure**.
4. Select the sheet from the Excel file that has been uploaded as a sample, as shown in the image below:

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/select-schema-elements-for-imm.png" class="no-border" >}}

5. Select the columns you want to import. You can either select all the columns by choosing **Check all** or select individual columns.
6. Click **OK** to create the **Import Mapping**.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/create-import-mapping.png" class="no-border" >}}

7. You can now choose to:
1. Map an existing entity by dragging and dropping an entity from your domain model via the **Connector** tab
2. Click **Map automatically** to create a new NPE in your domain model:

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/map-automatically.png" class="no-border" >}}

8. If you choose **Map automatically**, you can open the domain model and change the entity's name and persistence, as per your requirement.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/entity-name-persist-change.png" class="no-border" >}}

### Import With Mapping Activity in a Microflow {#import-with-mapping-MF}

As the Data Importer document contains a structure that is used as a source for import mapping, you can leverage the **Import with mapping** microflow activity to import data from input file(s).

The instructions below show how to import data from an Excel (or CSV) file.

1. Create a new microflow with a parameter (FileDocument) and drag the **Import with mapping** activity into it.
2. Double-click the activity.
3. In the **Input** section, in the **Variable** field, select an input file (Excel or CSV).
4. Select the mapping in the **Import Mapping** section.
5. Select **Range** and **Commit** options, if needed.
6. In the **Output** section, you can choose **Store in variable**, then click **OK**.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/import-with-mapping-params.png" class="no-border" >}}

### Before Running Your App {#data-import-with-mapping-app}

Before you can run your app, do the following:

1. Complete the microflow to show a page containing the entities committed after the import activity.
2. Call this microflow from a button on another page where a *FileDocument* object is created and has a provision to upload an input file.
3. Run your app locally and provide a file which is exactly like the sample file you have uploaded while creating this Data Importer document, then trigger the microflow.

You should see the data from the file being imported and shown on the page's data grid.

​ {{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/data-imported-from-input-file.png" class="no-border" >}}

## Edit Data Importer Document

You can edit the Data Importer document by uploading a new sample file.

{{% alert color="warning" %}}This action will erase the existing mapping or structure elements created for this document, and will replace it with new mapping and structure elements.{{% /alert %}}

### Upload a New File {#edit-DI-document-using-new-file}

To edit the Data Importer document, do the following:

1. Double-click the Data Importer document that you want to **Edit**. The document opens in read-only mode.
2. Click **Update File** in the top-right corner. You will receive notification that when a new file is uploaded and changes are saved, existing mapping/structure elements will be erased and will be replaced by new mapping/structure.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/update-data-importer-doc-confirmation.png" class="no-border" >}}

3. Click **Update**, then upload the new file.
4. Change the configuration, such as **Sheet Name** and **Header Row**.
5. Click **Create Structure** to update the document.

{{< figure src="/attachments/appstore/platform-supported-content/modules/data-importer-extension/data-importer-doc-updated.png" class="no-border" >}}

Similar steps can be followed to update the Data Importer document, which was created with **Implicit Mapping** in the [Creating a Data Importer Document](#create-document-with-imm) section.

You can now update the domain model entities, microflows, pages, and any other documents used or referenced by this Data Importer document to reflect the changes in your app.

## Known Issues

### Unchecked Columns
Expand Down