From 5fe8f7b74ef8615deb6df53f00d88e178c7103b7 Mon Sep 17 00:00:00 2001 From: Adelaide Nxumalo <27953420+anxumalo@users.noreply.github.com> Date: Mon, 14 Oct 2024 12:57:14 +0100 Subject: [PATCH] [DOCS-7529] Final UA review --- .../latest/admin/admin-properties.md | 50 ++-- federation-services/latest/admin/index.md | 92 +++---- federation-services/latest/admin/logging.md | 30 +- federation-services/latest/config/index.md | 257 ++++++++++-------- federation-services/latest/index.md | 8 +- federation-services/latest/install/index.md | 68 +++-- federation-services/latest/support/index.md | 9 +- federation-services/latest/using/index.md | 206 ++++++++------ 8 files changed, 402 insertions(+), 318 deletions(-) diff --git a/federation-services/latest/admin/admin-properties.md b/federation-services/latest/admin/admin-properties.md index b6805b9d9c..75117d8b8a 100644 --- a/federation-services/latest/admin/admin-properties.md +++ b/federation-services/latest/admin/admin-properties.md @@ -2,42 +2,36 @@ title: Admin Properties --- -## Admin Properties - -The Admin Properties section of Federation Services Admin allows a user to Configure the following Global, Analysis, and Internal Audits Properties. Once changes have been made to any of the properties use the **Update Configuration** button to save the changes to the properties file. +The Admin Properties section of Federation Services Admin allows you to configure the following Global, Analysis, and Internal Audits properties. Once changes have been made to any of the properties use the **Update Configuration** button to save the changes to the properties file. ## Global Properties -* Initialise Database on start-up. This will rebuild the database indexes as well as add or rename the connectors - this is the same as setting simflofy.initialize.mongo=true. -* Import database items - this is the same as setting simflofy.initialize.bootstrap=true -* Run various patches which will update the items in the database from 2.X to 3.X forms - this is the same as setting simflofy.initialize.update=true -* Enter the Full Discovery UI URL where these changes are to take place. For example: `https://localhost:8080/Federation-Services-discovery` +* Initialize Database on start-up. This will rebuild the database indexes as well as add or rename the connectors - this is the same as setting `simflofy.initialize.mongo=true`. +* Import database items - this is the same as setting `simflofy.initialize.bootstrap=true`. +* Run various patches which will update the items in the database from 2.X to 3.X forms - this is the same as setting `simflofy.initialize.update=true`. +* Enter the Full Discovery UI URL where these changes are to take place. For example: `https://localhost:8080/Federation-Services-discovery`. -Check the boxes and or set the intervals for the properties you want to update and use the update configuration button to push the changes to your Federation Services System. +Check the boxes and or set the intervals for the properties you want to update and use the **Update Configuration** button to push the changes to your Federation Services system. ## Analysis -The Analysis section of Federation Services Admin allows a user to set the following analysis tools for their Federation Services system. +The Analysis section of Federation Services Admin allows you to set the following analysis tools for their Federation Services system: -* Turn on the ability to Collect Data -* Allow users to generate Scheduled Reports -* Set time Interval between job check status -* Set Time between Schedule Status checks -* Set time of delay before the data collections service begins checking collection schedules +* Turn on the ability to Collect Data. +* Allow users to generate Scheduled Reports. +* Set time Interval between job check status. +* Set Time between Schedule Status checks. +* Set time of delay before the data collections service begins checking collection schedules. -Check the boxes for the properties you want to set and use the update configuration button to push the changes to your Federation Services System. +Check the boxes for the properties you want to set and use the **Update Configuration** button to push the changes to your Federation Services system. ## Internal Audits -By default, internal auditing is turned off. To turn auditing on check the box for Internal audits. Here you can choose to **Track All Internal Audits** by checking the Track all Audits box, or you can **Track Internal Audits by Type** by checking the box next to each action type you would like to audit. - -List of available Audit types: - -mappings, repository, job, job group, license, output, authentication, view, user, search, content search, widget, template, configuration, license +By default, internal auditing is turned off. See the [configuration](#configuration) section for more details. ### Overview -The internal audit table displays internal audits, each with a name, type, action, user, org, date, and a description. By default, the audits are sorted by descending order on the date column (most recent audits show at the top). All the other columns except the Date column can be sorted alphabetically if you click on the header. The Date column is sorted by date and time. An up arrow indicates that the column is sorted in ascending order while a down arrow indicates that the column is sorted in descending order. +The internal audit table displays internal audits, each with a name, type, action, user, org, date, and a description. By default, the audits are sorted in descending order on the date column (most recent audits show at the top). All the other columns except the Date column can be sorted alphabetically if you click on the header. The Date column is sorted by date and time. An up arrow indicates that the column is sorted in ascending order while a down arrow indicates that the column is sorted in descending order. A system admin will be able to see audits from all orgs while org admins will be able to see only their orgs audits. @@ -49,9 +43,9 @@ Below the table you will find search bars and select inputs. These allow you to By default all audits are selected. You can use multiple column filters to get more precise results. -The input does not have to be an exact match, for example if I want to get all audits that contain the word Filesystem then you could just enter File into the search bar. +The input does not have to be an exact match, for example if you want to get all audits that contain the word `Filesystem` then enter `File` into the search bar. -An example of filtering by column is shown below. Here we are filtering on the first three columns, Name, Type, and Action. + ### Pagination @@ -63,13 +57,13 @@ The date range picker allows you to look at audits from within a certain time pe Clicking on the calendar icon will bring up the current dates month and year. You can select a day on this month, or you can go to a different month by using the back arrow which goes to the previous month, or the forward arrow which goes to the next month. -Alternatively, you can just enter the date manually by selecting in the text field and entering a date and time that follows the format dd/mm/YYYY h:mm. +Alternatively, you can just enter the date manually by selecting in the text field and entering a date and time that follows the format `dd/mm/YYYY h:mm`. -For example, 12/23/2011 12:03 PM would be a valid date and time +For example, `12/23/2011 12:03 PM` would be a valid date and time ### Displayed Number of Entries -You can change the amount of audit are displayed using the number of entries selector. By default, the number of entries is set to 10. You can change the number of entries per page by clicking on the selector and selecting a different value. The options are 10, 25, 50, and 100. +You can change the number of audit entries displayed using the number of entries selector. By default, the number of entries is set to `10`. You can change the number of entries per page by clicking on the selector and selecting a different value. The options are `10`, `25`, `50`, and `100`. ### Configuration @@ -104,6 +98,6 @@ List of available Audit types: ### My Internal Audit -This page is available under the **Audit Reports** menu and will display a version of this page with information limited to the current user. +This page is available under the **Audit Reports** menu and displays a version of this page with information limited to the current user. -By default, the audits are sorted by descending order on the date column (most recent audits show at the top). All the other columns except the Date column can be sorted alphabetically if you click on the header. The Date column is sorted by date and time. An up arrow indicates that the column is sorted in ascending order while a down arrow indicates that the column is sorted in descending order. \ No newline at end of file +By default, the audits are sorted in descending order on the date column (most recent audits show at the top). All the other columns except the Date column can be sorted alphabetically if you click on the header. The Date column is sorted by date and time. An up arrow indicates that the column is sorted in ascending order while a down arrow indicates that the column is sorted in descending order. diff --git a/federation-services/latest/admin/index.md b/federation-services/latest/admin/index.md index a3999307fe..59bcebdd2d 100644 --- a/federation-services/latest/admin/index.md +++ b/federation-services/latest/admin/index.md @@ -4,63 +4,47 @@ title: Administer Federation Services The following diagram shows the architecture of Alfresco Federation Services: -![Architecture Diagram]({% link federation-services/images/architecture-diagram.png %}) +![Architecture Diagram]({% link federation-services/images/architecture-diagram.png %}){:height="395px" width="384px"} ## Configuration Tools Federation Services's configuration tool allows for easy import/export of all configurations in Federation Services. This is great for backups or for bootstrapping Federation Services with common configurations. -**Export Configurations** - -Through the User Interface on the left-hand menu under the Admin section select Configuration Tools. - -Click on the Export button. - -Select all the configuration you wish to export from the list. - -Select Export Configuration. This will trigger the download of a JSON file that contains all the selected configuration. - -**Import Configuration** - -Access the Configuration Tools menu from the Admin panel on the left-hand menu. Click on the Import button. - -Click Choose File and select the exported JSON file from - -Click Import. - ### Admin Tools -In the Admin Tools (Configuration Tools) Page users can import configurations, export configurations, export configs by job, manage patches and reset configurations. +In the **Admin Tools > Configuration Tools** page, users can import configurations, export configurations, export configs by job, manage patches and reset configurations. #### Export Configuration -Through the User Interface on the left-hand menu under the **Admin** section select **Admin Tools**. - -Click on the **Export** button. +Using the left-hand menu in the UI: -Select all the configuration you wish to export from the list. +1. Select **Admin Tools** under the **Admin** section. +2. Click on the **Export** button. +3. Select all the configuration you wish to export from the list. +4. Hit **Export Configuration**. -Hit **Export Configuration**. This will trigger the download of a JSON file that contains all the selected configuration. + This will trigger the download of a JSON file that contains all the selected configuration. #### Import Configuration -Access the **Configuration Tools** menu from the Admin panel on the left-hand menu. Click on the **Import** button. +Using the left-hand menu in the UI: -Click **Choose File** and select the exported JSON file from - -Click **Import**. +1. Access the **Configuration Tools** menu from the Admin panel on the left-hand menu. +2. Click on the **Import** button. +3. Click **Choose File** and select the exported JSON file from. +4. Click **Import**. ## Licenses The **Licenses** page allows for management of your License Keys. You can add a new License key, or reactivate previously entered keys. The details of each key are listed, including the associated MAC Address, Documents Allowed, Documents Used, End Date and which key is in use. -**Add a License Key** +### Add a License Key To add a license key simply paste the key into the text box labelled License Key and click Add License Key. You can only have one active license at a time. -**Reactivate a License Key** +### Reactivate a License Key -To reactivate a license key or adding a new license key copy the key and put it into the License Key text box, then click Add License Key. If the key already exists then it will be activated and the old key will be set to inactive. +To reactivate a license key or add a new license key, copy the key and put it into the `License Key` text box, then click **Add License Key**. If the key already exists then it will be activated and the old key will be set to inactive. ## Users @@ -70,9 +54,9 @@ Users can be created within Federation Services for user authentication and auth * Create, edit, delete or disable users. * View users by organization. * Manage active sessions to see who is currently logged in. From here you can end all sessions as well. For example, if you need to make updates to the system. -* And you can use the search field to narrow down the list of users displayed by name, email, login, or role +* You can use the search field to narrow down the list of users displayed by name, email, login, or role. -**Creating New Users** +### Creating New Users To create new users click on the **Create New User** button. On the **Edit User** page fill in the following details for the user you are creating: @@ -85,7 +69,7 @@ To create new users click on the **Create New User** button. On the **Edit User* ### List of User Role Definitions -**Federation Services Admin** +#### Federation Services Admin * **Discovery Only**: This user does not have access to the Federation Services Admin UI, and can only log in to Discovery * **Monitor**: Can **monitor** the execution of jobs, run reports, view job details. This is a READ ONLY user. @@ -94,9 +78,9 @@ To create new users click on the **Create New User** button. On the **Edit User* * **Org Admin**: Can **create, update, execute and delete** jobs, users and reports for a given Tenant. * **Federation Services Admin**: This user is the Federation Services Root user, with all capabilities on the system, for all tenants. -**Discovery** +#### Discovery -* **Discovery User**: Can log in to Discovery but cannot edit or delete views or alter configuration +* **Discovery User**: Can log in to Discovery but cannot edit or delete views or alter configuration. * **Discovery Admin**: Full access to basic federation features. ## User Groups @@ -104,15 +88,15 @@ To create new users click on the **Create New User** button. On the **Edit User* User groups are collections of users. They can be used as part of View Level Permissions. * While adding users you will be able to see their first name, last name, login and role. -* Federation Services Admins will be able to see all users +* Federation Services Admins will be able to see all users. * Organization Admins will only be able to see users from their organization. -* Includes the ability to search all users +* Includes the ability to search all users. ## Themes Theming gives an admin the ability to customise the Logo and Top bar colour as well as customise Lead Text sections throughout the application. Each organization may have a separate theme that can be set up by that organisations' admin. This is done in two parts. First, the Logo and Top bar colour, through the themes' page under the applications' admin section. Second, the Lead Text Sections, through the editing of a messages.properties file. -**Theme Fields** +### Theme Fields * **Browser Title**: The title that's displayed in the header * **Reset Browser Title**: Check to reset the browser title @@ -126,7 +110,7 @@ After selections, click Save. After page reloads, the changes will have taken ef The Active Jobs page shows the jobs that are currently running. Sometimes jobs can get stuck in the RUNNING state. -**Fixing Stuck Jobs** +### Fixing Stuck Jobs If a job is running and the **Abort** does not stop the job in an appropriate amount of time, the job can be killed manually through an Admin page. @@ -138,19 +122,15 @@ Once the job is killed, you will need to fully restart the job to run it again. The **Connectors** page shows the available Connector types with the option to Activate or Deactivate any given connector. Custom Connectors will be listed here as well. -## Organisation - -The Organization page shows the different Organisation used within the Federation Services platform for multi-tenancy. It will display the Organization name, short name, the status (Active/Inactive) which notes whether an organization is enabled, and also provide a link to search their users. - -### Overview +## Organization -The Organization page shows the different Organisation used within the Federation Services platform for multi-tenancy. It will display the Organization name, short name, the status (Active/Inactive) which notes whether an organization is enabled, and also provide a link to search their users. +The Organization page shows the different Organisations used within the Federation Services platform for multi-tenancy. It will display the Organization name, short name, the status (Active/Inactive) which notes whether an organization is enabled, and also provide a link to search their users. By default, Federation Services has one organization, based on configuration in the global properties. -**Tip:** This page assumes **simflofy.multi.tenant** is set to true in your global properties +> **Tip:** This page assumes `simflofy.multi.tenant` is set to `true` in your global properties. -In multi-tenant mode, a user with the Federation Services Admin role will have access to the **Organisation** page under the **Admin** menu. This page will display all the current tenants in your Federation Services. It will display whether an organization is enabled, and also provide a link to search their users. +In multi-tenant mode, a user with the Federation Services Admin role will have access to the **Organisation** page under the **Admin** menu. This page displays all the current tenants in your Federation Services. It will display whether an organization is enabled, and also provide a link to search their users. #### Creating an Organization @@ -160,18 +140,22 @@ When clicking on **Create New Organization** a pop-up will appear with the follo * **Organizational Key**: The org key that will follow the @ after users names. This cannot be changed after the organization has been creation. * **Organization Database**: This will be created and populated on submission. It cannot be changed after the organization has been creation. -The Key and Database fields cannot contain spaces or of the following characters +The Key and Database fields cannot contain spaces or of the following characters: -`!@#$%^&*()+=-[]\';,./{}|":<>?` +```text +!@#$%^&*()+=-[]\';,./{}|":<>? +``` Clicking **Continue** will take you to the org edit screen, but the org will not be created yet. You will be presented with a list of products available on your license, which you can activate for the new organization. This can be changed at any time to remove access to certain connectors and features. Upon clicking **Save** in this page, the organization will be added to the database and a standard initialisation will be performed (generation of Mongo indexes, adding the enabled connectors, and loading the bootstrap). -Users under non-global organisations will need to log in with +Users under non-global organisations will need to log in with: -`usersname@orgkey` +```text +usersname@orgkey +``` ## About Federation Services -On the About page users can find the version and build information for the installed version of Federation Services. \ No newline at end of file +On the **About** page users can find the version and build information for the installed version of Federation Services. diff --git a/federation-services/latest/admin/logging.md b/federation-services/latest/admin/logging.md index 766d83180b..83b9469fe3 100644 --- a/federation-services/latest/admin/logging.md +++ b/federation-services/latest/admin/logging.md @@ -2,22 +2,26 @@ title: Logging --- -Federation Services logging allows a user to access and set the log levels of all Federation Services classes. +Federation Services logging allows you to access and set the log levels of all Federation Services classes. ## Accessing and Updating the Logging File Federation Services logging is set in the following file: -`[Federation Services folder]\WEB-INF\classes\log4j.properties` +```text +[Federation Services folder]/WEB-INF/classes/log4j.properties +``` You can set the log level of all Federation Services classes by changing the following: -`log4j.logger.com.simflofy = debug, 3sixty-admin` +```text +log4j.logger.com.simflofy = debug, 3sixty-admin +``` Replace debug with another valid options: | Level | Description | -| ---|--- | +| ----- | ----------- | | ALL | All levels including custom levels. | | DEBUG | Designates fine-grained informational events that are most useful to debug an application. | | ERROR | Designates error events that might still allow the application to continue running. | @@ -27,18 +31,16 @@ Replace debug with another valid options: | TRACE | Designates finer-grained informational events than the DEBUG. | | WARN | Designates potentially harmful situations. | -In production Federation Services suggests setting logging to the ERROR level. +In production, Federation Services suggests setting logging to the `ERROR` level. -You'll notice there are other entries, these entries will override the com.simflofy setting for the specific packages they reference. +You'll notice there are other entries. These entries will override the `com.simflofy` setting for the specific packages they reference. -**For example**: +For example, this sets the SharePoint connector logging to `TRACE`. The second line is necessary to prevent double-logging: -``` +```text log4j.logger.com.simflofy.connectors.sharepoint = trace, 3sixty-admin log4j.additivity.com.simflofy.connectors.sharepoint = false -`` - -This sets the SharePoint connector logging to trace. The second line is necessary to prevent double-logging. +``` ### Federation Services Admin Log View @@ -55,14 +57,14 @@ simflofy.admin.log.path=/my/log/directory/logs/3sixty-admin.log On this page the user can download the complete 3sixty-admin.log, as well as filter the last 5000 lines of the log file: | Level | Logs | -| ---|--- | +| ----- | ---- | | Info | Info, Error | | Debug | Info, Error, Debug | | Trace | All | | Error | Error only | -You can also set the value simflofy.max.log.size to prevent performance issues. Default is 10 MB. KB and GB are also valid sizes. +You can also set the value `simflofy.max.log.size` to prevent performance issues. Defaults to `10 MB`. KB and GB are also valid sizes. ## Log Levels -On the **Log Levels** page you can add or remove [log appenders](https://dzone.com/articles/log-appender-what-is-it-and-why-would-you-use-it){:target="_blank"}. Both of which are temporary. There is a list of all available connectors if you wish to modify the logging level for an individual connector. \ No newline at end of file +On the **Log Levels** page you can add or remove [log appenders](https://dzone.com/articles/log-appender-what-is-it-and-why-would-you-use-it){:target="_blank"}. Both of which are temporary. There is a list of all available connectors if you wish to modify the logging level for an individual connector. diff --git a/federation-services/latest/config/index.md b/federation-services/latest/config/index.md index 1f222dcfe5..70dab65ea2 100644 --- a/federation-services/latest/config/index.md +++ b/federation-services/latest/config/index.md @@ -4,7 +4,13 @@ title: Configure Federation Services [Connectors](#connectors) are the 1st step in the integration process. They represent how Federation Services connects to other systems. -Connections are instances of connectors which the user configures. There are 5 types of connections: Authentication, Discovery, Integration, Content Service, and Content Search. +Connections are instances of connectors which the user configures. There are 5 types of connections: + +* Authentication +* Discovery +* Integration +* Content Service +* Content Search The Federation Services Wizards will help you quickly create all the connections needed for your data integration. Available Wizards: [Quick Build](#quick-build-wizard), [Integration](#integration-wizard), [Federation](#federation-wizard) @@ -204,13 +210,13 @@ Content Service Connections offer public REST endpoints that allow for integrati #### Basic Configuration -> **Tip**: Connector ids are how Federation Services identifies the individual connector when receiving calls from other sources. This value must be usable as part of url. Use the description field if you need more than a few letters/numbers to describe the connection. The description shows up with its connector ID across the product. +> **Tip:** Connector ids are how Federation Services identifies the individual connector when receiving calls from other sources. This value must be usable as part of url. Use the description field if you need more than a few letters/numbers to describe the connection. The description shows up with its connector ID across the product. * Connector ID: A unique identifier for this connection (Alphanumeric, dashes and underscore characters only) * Description: The text that will be displayed on drop-downs etc. to identify this connection. * Type: The type of Search Connection (Solr, Mongo, Elastic etc.) * Keep Connection Alive: Federation Services will cache the connection for a given amount of time before discarding it. -* Keep Alive in Milliseconds: How long to keep the connection alive before discarding it (300000 is 5 minutes) +* Keep Alive in Milliseconds: How long to keep the connection alive before discarding it (`300000` is 5 minutes) * Security Mode: This is how to authenticate with the back-end search. * Authentication Connection: The most common method is to use the appropriate authentication connection * User Pass-through Credentials: Users the authenticates with whatever authentication they used for Federation Services. Only supported in rare cases. @@ -273,11 +279,11 @@ Content View Connectors or Content Search Connector allow you to search content Connector ids are how Federation Services identifies the individual connector when receiving calls from other sources, such as Federation Service. This value must be usable as part of url. Use the description field if you need more than a few letters/numbers to describe the connection. The description shows up with its connectorId across the product. -* **Connector ID**: A unique identifier for this connection i.e. simflofy_demo (Alphanumeric, dashes and underscore characters only) +* **Connector ID**: A unique identifier for this connection i.e. `simflofy_demo` (alphanumeric, dashes and underscore characters only) * **Description**: The text that will be displayed on drop-downs etc. to identify this connection. * **Type**: The type of Search Connection (Solr, Mongo, Elastic etc.) * **Keep Connection Alive**: Federation Services will cache the connection for a given amount of time before discarding it. -* **Keep Alive in Milliseconds**: How long to keep the connection alive before discarding it (300000 is 5 minutes) +* **Keep Alive in Milliseconds**: How long to keep the connection alive before discarding it (`300000` is 5 minutes) * **Security Mode**: This is how to authenticate with the back-end search. * **Authentication Connection**: The most common method is to use the appropriate authentication connection * **User Pass-through Credentials**: Users the authenticates with whatever authentication they used for Federation Services. Only supported in rare cases. @@ -461,7 +467,7 @@ SharePoint Connection: When mapping "Modified by" field of SharePoint, schema in *For the MS Graph Mail connector and the Exchange Email connectors -In a job MS Graph Mail connector / Exchange Email connector to ECM with the job type set as manage in place, if the user changes the object type to E-Mail Message type through mapping, then when running the job, the external records will generate in ECM with object type as E-Mail Message and the default metadata fields set in the E-Mail Message will not get a value. +In a job MS Graph Mail connector / Exchange Email connector to ECM with the job type set as Manage in Place, if the user changes the object type to E-Mail Message type through mapping, then when running the job, the external records will generate in ECM with object type as E-Mail Message and the default metadata fields set in the E-Mail Message will not get a value. ##### Content Service Connection @@ -547,7 +553,7 @@ A Content View Connector defines the who, what and how of search. A better term Note that the usual `{"query":{}}` wrapper is not present. Including it will cause an error on search. -* **elastic_q**: This parameter, added by clicking **Add Custom Parameter**, allows a user to pass in a JSON formatted query to the elastic search server. When using this query method you must replace double quotes with single quote characters. +* **elastic_q**: This parameter, added by clicking **Add Custom Parameter**, allows you to pass in a JSON formatted query to the elastic search server. When using this query method you must replace double quotes with single quote characters. Here is an example query: `{'bool':{'must':[{'match':{'document_type':'accounting'}},{'match':{'account_type':''}}]}}` @@ -631,7 +637,7 @@ This section covers the specific configuration of the Content Service Connector. #### MongoDB -Can be used as a content search connector for Manage in Place +Can be used as a content search connector for Manage in Place. MongoDB is a source-available cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas. @@ -649,7 +655,7 @@ Authentication Configuration Fields: * **Use MongoDB GridFS Services**: Using MongoDB GridFS Services. Leave unchecked for regular MongoDB MONGO URI -Federation Services inserts the username and password into the connection string. In order to include them as part of the uri we use `[[USER]]:[[PASS]]` +Federation Services inserts the username and password into the connection string. In order to include them as part of the URI we use `[[USER]]:[[PASS]]` ##### Discovery Connector @@ -736,7 +742,7 @@ rd.setDenyAcl(deny); #### MongoDB GridFS -Can be used as a content search connector for Manage in Place +Can be used as a content search connector for Manage in Place. > **Note:** In Federation Services, the GridFS is just a mode of the MongoDB connector. @@ -758,7 +764,7 @@ GridFS by default uses two collections, [collection].files and [collection].chun * **Use MongoDB GridFS Services?**: Required for binary storage. This checkbox enables GridFS services MONGO URI -Federation Services inserts the username and password into the connection string. In order to include them as part of the uri we use `[[USER]]:[[PASS]]`. +Federation Services inserts the username and password into the connection string. In order to include them as part of the URI we use `[[USER]]:[[PASS]]`. ##### Integration Connection @@ -847,7 +853,7 @@ To add a default query which filters out all `.txt` documents: #### Alfresco -Can be used as an output connector for Manage in Place +Can be used as an output connector for Manage in Place. The Alfresco connector only operates in write mode. For read operations from Alfresco, use the CMIS connector. @@ -933,7 +939,7 @@ Meaning that if field 1 or field 2 is not present, do not add the aspects. Index files and emails from Google Drive and Gmail -> **Tip**: Google Drive folders usually display the ID in the url while looking at them in the browser. The root folder ("My Drive") does not show its ID, however. In order to retrieve it, you will need to create a Content Service Connection and use it to retrieve the id using the following url in your browser. If the root folder id it set on the content service connection, that folder's information will be returned. If blank, the My Drive folder's information will be returned. +> **Tip:** Google Drive folders usually display the ID in the url while looking at them in the browser. The root folder ("My Drive") does not show its ID, however. In order to retrieve it, you will need to create a Content Service Connection and use it to retrieve the id using the following url in your browser. If the root folder id it set on the content service connection, that folder's information will be returned. If blank, the My Drive folder's information will be returned. `3sixty-admin/api/repo/{googleConnectorId}/rootfolderid` ##### Authentication Connection @@ -984,22 +990,27 @@ Click **Add Scope** and check **Creating OAuth Credentials** -1. Return to the [APIs and Services Dashboard](https://console.cloud.google.com/apis/dashboard){:target="_blank"} -2. Click **Credentials** -3. Click **Create Credentials** -4. Click OAuth Client Id -5. On the next screen, select **Web Application** -6. Give the credentials a name -7. Under **Authorised Redirect URIs**, add the following - ``` +1. Return to the [APIs and Services Dashboard](https://console.cloud.google.com/apis/dashboard){:target="_blank"}. +2. Click **Credentials**. +3. Click **Create Credentials**. +4. Click OAuth Client Id. +5. On the next screen, select **Web Application**. +6. Give the credentials a name. +7. Under **Authorised Redirect URIs**, add the following: + + ```text http://{SIMFLOFY_SERVER}/3sixty-admin/authconn/oauthcb ``` + For example. If you're running Federation Services on a local machine `http://localhost:8080/3sixty-admin/authconn/oauthcb` - **Important:** For Manage In Place the domain server has to be public server since Google Drive only supports public server or localhost - **Tip:** 127.0.0.1 will not work, but **localhost** will. -8. Click **Create** -9. Your new credentials will appear under **OAuth 2.0 Client IDs**. Click the Download button on the right and retrieve the Client ID and Client Secret from the downloaded json file. + + > **Important:** For Manage In Place, the domain server has to be public server since Google Drive only supports public server or localhost. + + > **Tip:** `127.0.0.1` will not work, but `localhost` will. + +8. Click **Create**. +9. Your new credentials will appear under **OAuth 2.0 Client IDs**. Click the **Download** button on the right and retrieve the Client ID and Client Secret from the downloaded JSON file. **Creating your connector in Federation Services** @@ -1038,16 +1049,18 @@ Use this method if you wish to create a service user to interact with the data. **Creating The Service Account** -1. Go to the [APIs and Services Dashboard](https://console.cloud.google.com/apis/dashboard){:target="_blank"} -2. Click **Credentials** -3. Click Create Credentials -4. Select Service Account +1. Go to the [APIs and Services Dashboard](https://console.cloud.google.com/apis/dashboard){:target="_blank"}. +2. Click **Credentials**. +3. Click Create Credentials. +4. Select Service Account. 5. You will be asked you to assign roles to the account. Since we need wide-ranging access for now, we should choose Owner. This can be changed later. 6. The final screen invites other users to access this service account through their own service account portals. Use as needed for your organization. 7. Click **DONE**. You will be taken back to the service account list for the project. -**Tip:** Make note of the email address of this user. Any folders or Shared Drives you wish to migrate will need to be shared with this user. -8. Click on the three dots under Actions, then click **Manage Keys** -9. Click **ADD KEY** then **Create New Key**, then select **JSON** + + > **Tip:** Make note of the email address of this user. Any folders or Shared Drives you wish to migrate will need to be shared with this user. + +8. Click on the three dots under Actions, then click **Manage Keys**. +9. Click **ADD KEY** then **Create New Key**, then select **JSON**. 10. Click **Create**. You will download a json file to be used in the next step. ###### Creating your JWT Auth Connector in Federation Services @@ -1110,7 +1123,7 @@ google site - txt However, these new ones don't seem to support revisions so we can't get the revisions when getVersion is checked! -**Tip:** If a document of the same name exists within a parent folder, the connector will attempt to create a new revision of the document, instead of uploading a new one. These revisions have the `keepRevisionForever` flag set to `true`, meaning Google Drive will not delete them automatically, as it does with most revisions. A document can have a maximum of 200 revisions. +> **Tip:** If a document of the same name exists within a parent folder, the connector will attempt to create a new revision of the document, instead of uploading a new one. These revisions have the `keepRevisionForever` flag set to `true`, meaning Google Drive will not delete them automatically, as it does with most revisions. A document can have a maximum of 200 revisions. ##### Run and Monitor Jobs @@ -1207,7 +1220,7 @@ rd.setACL(newAcl); ##### Interacting with Shared Drives (3.1.2+) -**Tip:** These cases assume that you are working with a Google Workspace and the credentials you supply are allowed to view / access Shared Drive information. +> **Tip:** These cases assume that you are working with a Google Workspace and the credentials you supply are allowed to view / access Shared Drive information. **Get a Shared Drive Id** @@ -1260,7 +1273,7 @@ The Tenant, Client ID, and Client Secret, are all provided to you during your Ap ##### Authentification Connection -> **Tip**: Tip: Its recommended that OAuth be used for Migration only. And Client / Secret Auth be used for Content Service +> **Tip:** Tip: Its recommended that OAuth be used for Migration only. And Client / Secret Auth be used for Content Service This connector requires a standard [Microsoft Graph](#microsoft-graph) Authentication Connection. @@ -1412,7 +1425,9 @@ Note: The folderId can be a path off of the root library. Such as /test. ###### Update File Content -`PUT /api/repo/graph/updateContent?fileId=01WNAC6Z32P4QVKHLA7ZC2SZH7IYTSDON7` +```text +GET /api/repo/graph/updateContent?fileId=01WNAC6Z32P4QVKHLA7ZC2SZH7IYTSDON7 +``` > Note: Set the new content as the request body @@ -1454,7 +1469,9 @@ Note: The folderId can be a path off of the root library. Such as /test. ###### Update Files Properties -`PUT /api/repo/graph/updateProperties?fileId=01WNAC6ZYYYWDZOWH2DFH3LRHT7MWF5L2R&3SixtyText=metafield` +```text +GET /api/repo/graph/updateProperties?fileId=01WNAC6ZYYYWDZOWH2DFH3LRHT7MWF5L2R&3SixtyText=metafield +``` > Note: Each field will be passed as a separate parameter @@ -1469,7 +1486,9 @@ Note: The folderId can be a path off of the root library. Such as /test. ###### Folder Items -`PUT /api/repo/graph/folderitems?id=01WNAC6Z4BAYBXXIMILVBLXKCZ6K3VVNHJ` +```text +GET /api/repo/graph/folderitems?id=01WNAC6Z4BAYBXXIMILVBLXKCZ6K3VVNHJ +``` ```text { @@ -1671,11 +1690,11 @@ or #### Microsoft Graph Teams -> **Important**: Due to the potentially sensitive nature of Teams' content, Microsoft has restricted the usage of APIs that can retrieve chat messages. In order to use these APIs, an additional approval process is required. The form is found [here](https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR1ax4zKyZjVBmutzKVo1pVtUQ1VJMlNTNUdJV1FKTzVZSVU4MlMwTTdOTSQlQCN0PWcu){:target="_blank"}. The response will usually take 1 to 2 business days. +> **Important:** Due to the potentially sensitive nature of Teams content, Microsoft has restricted the usage of APIs that can retrieve chat messages. In order to use these APIs, an additional approval process is required. The form is found [here](https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR1ax4zKyZjVBmutzKVo1pVtUQ1VJMlNTNUdJV1FKTzVZSVU4MlMwTTdOTSQlQCN0PWcu){:target="_blank"}. The response will usually take 1 to 2 business days. -> **Important**: In order to retrieve private chat messages, a payment model is required after a certain number of messages. See [this link](https://learn.microsoft.com/en-us/graph/teams-licenses#evaluation-mode-default-requirements){:target="_blank"}. Federation Services uses the Get messages across all chats for user method. +> **Important:** In order to retrieve private chat messages, a payment model is required after a certain number of messages. See [this link](https://learn.microsoft.com/en-us/graph/teams-licenses#evaluation-mode-default-requirements){:target="_blank"}. Federation Services uses the Get messages across all chats for user method. -> **Note:** Microsoft Teams file storage for a team is actually just a SharePoint site, with each channel being a folder in the `Documents` library. You can view your teams files in SharePoint by going to `https://[tenant].sharepoint.com/sites/[teamName]`. +> **Note:** Microsoft Teams file storage for a team is actually just a SharePoint site, with each channel being a folder in the `Documents` library. You can view your Teams files in SharePoint by going to `https://[tenant].sharepoint.com/sites/[teamName]`. ##### Authentication Connection @@ -1798,7 +1817,10 @@ The name of this connector is **Microsoft Graph oAuth Connector** * **Service URL**: This should be set to : [https://login.microsoftonline.com](https://login.microsoftonline.com/) * **Scope**: These are the permissions required to access OneDrive files. Multiple permissions are separated by a space. Default scope: user.read files.readwrite.all -* **Tenant**: The directory tenant the application plans to operate against, in GUID or domain-name format. **Note**: When accessing a personal drive replace the guid with the word "common". +* **Tenant**: The directory tenant the application plans to operate against, in GUID or domain-name format. + +> **Note:** When accessing a personal drive replace the GUID with the word "common". + * **Client ID**: The client id of the application. * **Client Secret**: The client secret for the application. @@ -1820,7 +1842,10 @@ Works with Business and SharePoint OneDrive. Personal drives not supported at th * **Site Names**: A list of sites to crawl. **Only applicable if accessing a SharePoint Drive via a standard authenication connection** * **Query**: A query to run against each drive. For a Business drive, leave blank to retrieve all items in the drive. For a Personal drive you must enter at least one search term. -* **Process Folders**: Process folders as well as documents. **Note: Empty folders will be processed.** +* **Process Folders**: Process folders as well as documents. + +> **Note: Empty folders will be processed.** + * **Get Versions**: Retrieve document versions. * **Include Permissions**: Include the folder/document permissions. @@ -1877,11 +1902,11 @@ Only fill this is if your Exchange server uses LDAP Authentication * **LDAP Username**: Username to authenticate with and query LDAP. * **LDAP Password**: Password for the LDAP Username. (Note: password is encrypted and never displayed in the input form after the Save button is pressed) * **LDAP Domain**: Domain of the LDAP server. -* **LDAP Base Filter**: The base filter to search for your ldap user. Defaults to (&(&(objectCategory=Person)(objectClass=User))) +* **LDAP Base Filter**: The base filter to search for your ldap user. Defaults to `(&(&(objectCategory=Person)(objectClass=User)))`. #### Discovery Connector -Normally Federation Services Discovery is used to discovery the metadata models of underlying repository. In the case of Microsoft Exchange the Exchange Web Services (EWS) API provides us with the models as things such as Email Message and Calendar Event are static models depending on which version of the EWS you are using. +Normally Federation Services Discovery is used to discover the metadata models of the underlying repository. In the case of Microsoft Exchange, the Exchange Web Services (EWS) API provides us with the models as things such as Email Message and Calendar Event are static models depending on which version of the EWS you are using. It is still necessary to run Discovery in order to do Job mappings. You would only need to run Exchange Discovery Once and then use that Discovery for any Exchange Jobs. Using the Exchange Discovery, you can map Email Message fields to the output fields. @@ -1920,9 +1945,9 @@ Job specification includes Job run setting like number of worker threads and bat ##### Content Service Connection -The source repository ids for exchange are a compound id in the form of +The source repository ids for exchange are a compound id in the form: -sourceEmail_SID_exchangeUniqueId +`sourceEmail_SID_exchangeUniqueId` **Supported Methods** @@ -1959,10 +1984,10 @@ folder.isversion=CHECKBOX ``` SPACES -If a type has a space in it, replace it with the value \u0020. As an example: +If a type has a space in it, replace it with the value `\u0020`. As an example: `Historical\u0020Documents.field=CHECKBOX` -Here is a list of available field types +Here is a list of available field types: `CHECKBOX,DATETIME,TEXT,TEXTAREA,INTEGER,LONG,DECIMAL,DOUBLE,URI,READONLY,BINARY` @@ -1979,11 +2004,11 @@ For retrieving content and its associated metadata from the specified filesystem **Paths (Repo)** -Click the green plus sign below the paths to add more than one File Path +Click the green plus sign below the paths to add more than one File Path. Click the red minus sign next to a file path to remove it. -Manage In Place does not support multiple paths for the Filesystem Connector +Manage In Place does not support multiple paths for the Filesystem Connector. * **File Path**: The path of a folder to crawl. * **Convert to URI**: For Windows filesystems, converts backslashes to forward slashes and removes drive identifiers @@ -2014,7 +2039,7 @@ The methods currently supported for this connector are: **Connection Configuration** * **Root Path**: The location of your folders and files that you want Federation Services to access. Will be prepended to the id on all calls to the connection -* Note: For a content service connection make sure the Root folder in the content service connection is the same as the root folder in the job properties. +* Note: For a content service connection make sure the Root folder in the content service connection is the same as the root folder in the job properties. IDS Filesystem content services uses the files or folder's path as an ID. The connection will attempt to prepend the configured root folder on all calls. @@ -2037,7 +2062,9 @@ Depending on how many documents you found during the crawl > 100,000, you may ne Next we group by docHash and output to a new collection named duplicates (You can name the new output collection to anything you like). -`db.tsRecordProcessed.aggregate([{$group:{_id:"$docHash",docs:{$push:"$doc_id"},doc_names:{$push:"$doc_name"}}},{$project:{docs:1,doc_names:1,numDocs:{$size:"$docs"}}},{$match:{numDocs:{$gt:1}}},{$out:"duplicates"}])` +```text +db.tsRecordProcessed.aggregate([{$group:{_id:"$docHash",docs:{$push:"$doc_id"},doc_names:{$push:"$doc_name"}}},{$project:{docs:1,doc_names:1,numDocs:{$size:"$docs"}}},{$match:{numDocs:{$gt:1}}},{$out:"duplicates"}]) +``` You can now export de-dupes collection to CSV or JSON using mongoexport @@ -2088,7 +2115,7 @@ If you are using Amazon Glacier as an Output Connection you will need to fill ou **Output Specifications** * **Output Folder Path** : Path where archive zip file is created before sending to Glacier. -* **Glacier End Point** : Glacier end point where the vault exists. For example, `https://glacier.us-east-1.amazonaws.com` +* **Glacier End Point** : Glacier end point where the vault exists. For example, `https://glacier.us-east-1.amazonaws.com`. * **Glacier Vault** : Name of the vault to store archive files. ##### Content Service Connection @@ -2099,7 +2126,7 @@ This section covers the Glacier specific configuration of the Content Service Co This section covers the Glacier specific configuration of the Content Service Connector. -* **End Point**: Glacier end point where the vault exists. For example, `https://glacier.us-east-1.amazonaws.com` +* **End Point**: Glacier end point where the vault exists. For example, `https://glacier.us-east-1.amazonaws.com`. * **Vault Name**: Name of the vault to store archive files. **Supported Methods** @@ -2129,16 +2156,16 @@ Authentication connectors are used to authenticate repository/output connections * **Connection Timeout**: Set the connection timeout. Higher values may be needed when moving large files. > **Tip:** INSTALLED AWS CREDENTIALS -If you leave the Client ID and Client Secret empty, Federation Services will attempt to authenticate with your installed AWS credentials +If you leave the Client ID and Client Secret empty, Federation Services will attempt to authenticate with your installed AWS credentials. **Proxy Information** This tab is for if you're connecting through a proxy, and is optional. * **Proxy User**: The proxy user to use. (Optional) -* **Proxy Password**: The password for the proxy user. (leave blank if no proxy) +* **Proxy Password**: The password for the proxy user. (Leave blank if there's no proxy) * **Proxy Protocol**: The HTTP(S) Protocol to use to connect to the proxy. -* **Full Proxy Url**: The Proxy Host (leave blank if no proxy). +* **Full Proxy Url**: The Proxy Host. (Leave blank if there's no proxy) * **Proxy Port**: The port to connect to on the proxy. (Optional) * **Proxy Domain**: The Domain for the proxy. * **Proxy Workstation**: The workstation to use. @@ -2172,7 +2199,7 @@ Specification Tab: S3 Folders (Repo) * **Bucket Name**: The bucket name that will be prepended to all keys. * **Includes Unmapped Properties**: Will apply all metadata on the document without mapping * **Use GZip**: Sets whether gzip decompression should be used when receiving HTTP responses. -* **Do not generate XML when Outputting to S3**: Like the BFS Connector, the S3 Connector outputs metadata as separate files in the for of [filename].metadata.properties.xml. Check this box if you wish for it to only output files. +* **Do not generate XML when Outputting to S3**: Like the BFS Connector, the S3 Connector outputs metadata as separate files in the form of `[filename].metadata.properties.xml`. Check this box if you wish for it to only output files. * **Use Transfer Manager**: If migrating larger files, the S3 APIs offer a transfer manager to ensure more stable uploads * **Stage Binary to Filesystem**: To avoid issues with disconnects from the source, this will temporarily store file content in the Tomcat temp folder before uploading it. * **Date/DateTime Format**: How to format the mapped fields of this type before upload. @@ -2198,7 +2225,7 @@ This section covers the S3 specific configuration of the Content Service Connect This section covers the S3 specific configuration of the Content Service Connector. -> **Tip:** S3 file ids always take the form of /bucket/(key). +> **Tip:** S3 file ids always take the form of `/bucket/(key)`. * **Bucket Name**: The target bucket for creating a file. * **Output Folder Path**: The key of the folder to target when creating a file. @@ -2207,7 +2234,7 @@ This section covers the S3 specific configuration of the Content Service Connect **Supported Method** -* Create File - Will take full /bucket/key as folderId parameter to bucket and folder configuration +* Create File - takes full `/bucket/key` as `folderId` parameter to bucket and folder configuration * Delete Object by ID * Get File Content * Get Object Properties @@ -2314,8 +2341,8 @@ The Apache Kafka Authentication Connector allows you to read from an Apache Kafk **Apache Kafka Authentication Connection Fields** * **Name**: Unique name for the connection -* **Bootstrap Servers**: Comma Delimited list of servers (in host:port format) to consume from/publish to. Required. -* **Topic Name**: Topic name to publish to or read from. Required +* **Bootstrap Servers**: Comma Delimited list of servers (in `host:port` format) to consume from/publish to. Required. +* **Topic Name**: Topic name to publish to or read from. Required. * **SASL Protocol**: The security.protocol config option to set. Will not do anything if SASL Mechanism and SASL JAAS Config Fields are empty. Optional. * **SASL Mechanism**: The sasl.mechanism config option to set. Will not do anything if Security Protocol and SASL JAAS Config Fields are empty. Optional. * **SASL JAAS Config**: The sasl.jaas.config config option to set. Will not do anything if Security Protocol and SASL Mechanism Fields are empty. Optional. @@ -2335,7 +2362,7 @@ Most Integration Connections can act in both repository (read) and output (write ##### Job Configuration -A Federation Services Job is the process of moving or syncing content(including versions, ACL's, metadata) from one CMS (content management system) to another. +A Federation Services Job is the process of moving or syncing content (including versions, ACLs, metadata) from one CMS (content management system) to another. **Repository Configuration**: No additional job configurations needed to set up Kafka as a repository source **Output Configuration**: No additional job configurations needed to set up Kafka as an output source @@ -2343,6 +2370,7 @@ A Federation Services Job is the process of moving or syncing content(including #### Apache Solr Solr is an open-source enterprise-search platform, written in Java, from the Apache Lucene project. Its major features include full-text search, hit highlighting, faceted search, real-time indexing, dynamic clustering, database integration, NoSQL features and rich document handling. + > **Important:** While Federation Servicestion Services still supports Solr for some basic federation cases it will not receive any further enhancements beyond its current state. [More info on Apache Solr](https://solr.apache.org/){:target="_blank"} @@ -2350,15 +2378,16 @@ Solr is an open-source enterprise-search platform, written in Java, from the Apa ##### Authentication Connection Authentication connectors are used to authenticate repository/output connections that need certain authentication fields like access tokens or refresh tokens. Click here for more information on setting up an Authentication Connection. + > **Important:** Federation Services has no way of creating a Solr core or generating a Solr schema. The configured Solr core must already exist. For the simplest case, go to your Silurian directory and execute the command `solr -c create _core-name_`. Any attempts to authenticate without a valid core will fail. -**Configuration** +**Configuration** -* **Name:** The name of the connection -* **Host:** The location of the Solr server. Can support multiple urls, comma delimited, if using Solr Cloud +* **Name:** The name of the connection. +* **Host:** The location of the Solr server. Can support multiple urls, comma delimited, if using Solr Cloud. * **Solr Core Name:** The name of the Solr core (collection) to authenticate to. -* **Username:** (Optional) The username, if security is enabled -* **Password:** (Optional) The password, if security is enabled +* **Username:** (Optional) The username, if security is enabled. +* **Password:** (Optional) The password, if security is enabled. * **Use SolrCloud:** A special high-availability set up for clustered Solr servers. * **Use ZooKeeper:** Another Apache product used for clustering servers. @@ -2366,14 +2395,14 @@ Authentication connectors are used to authenticate repository/output connections The Solr Integration Connection is designed to index content into a Solr core -**Configuration** +**Configuration** * **Connection Name**: This is a unique name given to the connector instance upon creation. * **Description**: A description of the connector to help identify it better. ##### Job Configuration -A Federation Servicestion Services Job is the process of moving or syncing content (including versions, ACL's, metadata) from one CMS (content management system) to another. +A Federation Servicestion Services Job is the process of moving or syncing content (including versions, ACLs, metadata) from one CMS (content management system) to another. **Solr Output Specifications** @@ -2481,7 +2510,7 @@ The Aprimo Integration Connection has no configuration as we use a Federation Se ##### Job Configuration -A Federation Services Job is the process of moving or syncing content (including versions, ACL's, metadata) from one CMS (content management system) to another. +A Federation Services Job is the process of moving or syncing content (including versions, ACLs, metadata) from one CMS (content management system) to another. * **Repository Connection Configuration**: Aprimo cannot be used as a Repository Connection * **Output Connection Configuration**: No additional configuration needed to use Aprimo as an Output source @@ -2539,7 +2568,7 @@ In output mode, the connector will output documents in the same fashion as the [ ##### Job Configuration -A Federation Services Job is the process of moving or syncing content (including versions, ACL's, metadata) from one CMS (content management system) to another. +A Federation Services Job is the process of moving or syncing content (including versions, ACLs, metadata) from one CMS (content management system) to another. **Output Specification Fields** @@ -2832,7 +2861,7 @@ Box Secret Key: Secret Key Box will generate for you as part of the Application AUTHENTICATION -After filling in your client id and secret. Hit the "Authenticate" button. You will be redirected to a screen in Box asking you to confirm the application permissions. You should be returned to Federation Services after accepting. If you receive an error, your redirect uri in the Box Application config may not be correct. +After filling in your client id and secret. Hit the **Authenticate** button. You will be redirected to a screen in Box asking you to confirm the application permissions. You should be returned to Federation Services after accepting. If you receive an error, your redirect URI in the Box Application config may not be correct. ##### Discovery Connector @@ -3206,7 +3235,7 @@ Also known as an input connection. It's job is to query or crawl remote systems * Description: Provide a description for this connection * Type: Select the Filesystem Content Service Connector * Keep Connection Alive: Keep this connection active -* Keep alive in Milliseconds (300000 is 5 minutes): How long until connection expires if unused +* Keep alive in Milliseconds (`300000` is 5 minutes): How long until connection expires if unused * Connection URL: The web address for your connection * Security Mode: None needed for this connection * Mapping Type: Choose single map or group mapping if you are using mapping for jobs @@ -4265,7 +4294,7 @@ This setup is for CMOD ODWek version 9.5 (64 bit). Other versions of ODWek can a ##### Job Configuration -A Federation Services Job is the process of moving or syncing content (including versions, ACL's, metadata) from one CMS (content management system) to another. +A Federation Services Job is the process of moving or syncing content (including versions, ACLs, metadata) from one CMS (content management system) to another. **Repository Configuration Fields** @@ -4850,52 +4879,58 @@ This section covers the JDBC specific configuration of the Content Service Conne ##### Advanced Topic: Connecting to a Microsoft Access Database -To use an Access database you'll need a JDBC Driver [here](http://ucanaccess.sourceforge.net/site.html){:target="_blank"} +To use an Access database you'll need a JDBC Driver [here](http://ucanaccess.sourceforge.net/site.html){:target="_blank"}. -You will need to install the jars in the 3sixty-admin web app. [This post](https://stackoverflow.com/questions/21955256/manipulating-an-access-database-from-java-without-odbc/21955257#21955257){:target="_blank"} tells you the jars you will need: +You will need to install the JARs in the 3sixty-admin web app. [This post](https://stackoverflow.com/questions/21955256/manipulating-an-access-database-from-java-without-odbc/21955257#21955257){:target="_blank"} tells you the JARS you will need: -Place them into `3sixty-admin/WEB-INF/lib` +Place them into `3sixty-admin/WEB-INF/lib`. -After installing the jars you should be able to start Federation Services and use the JDBC connector as per usual. +After installing the JARs you should be able to start Federation Services and use the JDBC connector. **Example JDBC to BFS Oracle Integration** -This process will walk you through setting up an integration from a JDBC Repository to a BFS Output system. +This process walks you through setting up an integration from a JDBC Repository to a BFS Output system. 1. **Create a JDBC Auth Connector**: -For Oracle there are a couple of things to note: + For Oracle there are a couple of things to note: -* The username must be all caps -* The SID/Schema must be all caps as well -* The username can be different from the Schema name, but obviously needs permissions to that Schema. -* Note the Driver class + * The username must be all caps + * The SID/Schema must be all caps as well + * The username can be different from the Schema name, but obviously needs permissions to that Schema. + * Note the Driver class 2. **Create a JDBC Discovery Connector**: -Set the authentication connector to be the auth connector created above. For Oracle, the above will get all objects in your database. + Set the authentication connector to be the auth connector created above. For Oracle, the above will get all objects in your database. -Trouble Shooting: + Troubleshooting: -Remove any ojdbc jar file from the 3Sixty-admin/WEB-INF/lib directory and put in the ojdbc jar for your Oracle version. For Oracle 12c as an example that would be ojdbc8.jar. + Remove any ojdbc jar file from the 3Sixty-admin/WEB-INF/lib directory and put in the ojdbc jar for your Oracle version. For Oracle 12c as an example that would be ojdbc8.jar. -The schema pattern should be set to the schema you want. Oracle seems to hang if you leave that field blank. + The schema pattern should be set to the schema you want. Oracle seems to hang if you leave that field blank. -Table types can be a comma delimited list of the following: + Table types can be a comma delimited list of the following: -* TABLE -* VIEW -* SYSTEM TABLE -* GLOBAL TEMPORARY -* LOCAL TEMPORARY -* ALIAS -* SYNONYM + * TABLE + * VIEW + * SYSTEM TABLE + * GLOBAL TEMPORARY + * LOCAL TEMPORARY + * ALIAS + * SYNONYM 3. **Create an integration connector for JDBC**: -4. **Create an integration connector for BFS.** **NOTE**: BFS does not have an authentication connector. -5. **Create a job with the repo connector** as your jdbc integration connector and the output connector as your BFS integration connector: +4. **Create an integration connector for BFS.** + + > **Note:** BFS does not have an authentication connector. + +5. **Create a job with the repo connector** as your JDBC integration connector and the output connector as your BFS integration connector. + 6. **Edit the BFS Tab**: -Chose your output folder path that must exist ahead of time. Check the **Include Un-Mapped Properties** check box. Leave the rest as the default like the above picture. + + Chose your output folder path that must exist ahead of time. Check the **Include Un-Mapped Properties** check box. Leave the rest as the default like the above picture. + 7. **Edit the JDBC Tab**: ::: Configuration Some things to note about this configuration: @@ -5128,13 +5163,13 @@ The Mobius Integration Connection has no configuration as we use a Federation Se ##### Job Configuration -A Federation Services Job is the process of moving or syncing content (including versions, ACL's, metadata) from one CMS (content management system) to another. +A Federation Services Job is the process of moving or syncing content (including versions, ACLs, metadata) from one CMS (content management system) to another. * **Repository Connection Configuration**: No additional configuration needed to use Mobius as an Repository source **Output Specification** -* **Repository Name**: Mobius Repository Name (ie 'Mobius') +* **Repository Name**: Mobius Repository Name (ie 'Mobius'). * **Include Un-Mapped Properties**: All metadata will be output, even if no mappings are supplied. These values will be applied as global properties to the file. #### Nuxeo @@ -5149,13 +5184,13 @@ The Nuxeo Authentication Connector contains configuration to authenticate with a **Configuration Options** * **Name**: The name of the connector. -* **Nuxeo Username**: The name of the user to authenticate with Nuxeo -* **Nuxeo Password**: The password of the user authenticating with Nuxeo -* **Nuxeo URL**: The url to the **Nuxeo** endpoint +* **Nuxeo Username**: The name of the user to authenticate with Nuxeo. +* **Nuxeo Password**: The password of the user authenticating with Nuxeo. +* **Nuxeo URL**: The url to the **Nuxeo** endpoint. * **Use Caching**: (required) Should the client use caching when making authentication requests? -* **Session Timeout**: Maximum session timeout in ms -* **Transaction Timeout**: Maximum transaction timeout in ms -* **Nuxeo Provider ID**: The provider ID set in the Nuxeo's configuration for MIP (Manage-in-place/ Content federation). +* **Session Timeout**: Maximum session timeout in milliseconds (ms). +* **Transaction Timeout**: Maximum transaction timeout in milliseconds (ms). +* **Nuxeo Provider ID**: The provider ID set in the Nuxeo's configuration for Manage in Place (MIP) /Content Federation. ##### Discovery Connector @@ -5191,7 +5226,7 @@ The Nuxeo Content Service Connection provides a full ECM API for interacting wit **Basic Configurations** -* **Connector ID**: A unique identifier for this connection i.e. simflofy_demo (Alphanumeric, dashes and underscore characters only) +* **Connector ID**: A unique identifier for this connection i.e. `simflofy_demo` (alphanumeric, dashes and underscore characters only) * **Description**: The text that will be displayed on drop-downs etc. to identify this connection. * **Type**: The type of Search Connection (Solr, Mongo, Alfresco Faceted etc.) * **Connection URL**: Your server's web address @@ -5315,7 +5350,7 @@ The Authentication connection configuration is simple as it only contains three ##### Discovery Connector -The Salesforce Discovery Connector allows a user to connect to their Salesforce domain and extract information about available data types and metadata. +The Salesforce Discovery Connector allows you to connect to your Salesforce domain and extract information about available data types and metadata. **Discovery Configuration** diff --git a/federation-services/latest/index.md b/federation-services/latest/index.md index e3f2f8e949..947265a958 100644 --- a/federation-services/latest/index.md +++ b/federation-services/latest/index.md @@ -2,20 +2,20 @@ title: Alfresco Federation Services --- -Alfresco Federation Services is an add-on module that provides a powerful and easy way to search and manage content federated from leading business systems and content management applications. Manage in place functionality allows you to access, control, and govern content residing in more than sixty different business and content repository types. This provides a single view of information across different content systems, by synchronizing content into Alfresco Content Services. +Alfresco Federation Services is an add-on module that provides a powerful and easy way to search and manage content federated from leading business systems and content management applications. Manage in Place (MIP) functionality allows you to access, control, and govern content residing in different business and content repository types. This provides a single view of information across different content systems, by synchronizing content into Alfresco Content Services. By connecting information from different systems, you can provide a single view of information stored across multiple repositories. Below is a summary of the key capabilities: * Federated search - content can be searched for across multiple content repositories and made accessible inside Alfresco Content Services. This means it doesn't need to be migrated from different content systems. -* Manage content in place - all content can be controlled, no matter where it's stored. +* Manage content in place (MIP) - all content can be controlled, no matter where it's stored. * Intelligent content migration - enables content migration to be completed in the background, with minimal disruption to your end users. Alfresco Federation Services ensures that customers are able to: * Federate and manage content or records in other repositories -* Apply Alfresco Governance Services to content that's stored in other content systems (using the manage in place functionality) +* Apply Alfresco Governance Services to content that's stored in other content systems (using the Manage in Place functionality) * Support enterprise-wide eDiscovery cases in order to allow legal holds across the enterprise The following diagram shows a simple representation of how Alfresco Content Services and Alfresco Federation Services interact with different content systems. @@ -36,7 +36,7 @@ Here's some useful terminology from the Federation Services documentation. |Auth Connector|This allows you to authenticate against a repository.| |Repository Connector|This is a connector to a repository for getting content, metadata, versions, and renditions.| |Output Connector|This is a connector to the output system you want to migrate or index to. | -|Content Service Connector|This connector allows you to attach a system to the Federation Services Content Services API for Federation.| +|Content Service Connector|This connector allows you to attach a system to the Content Services API for Federation.| |Discovery Connector|This connector is used to get schema information from a system.| |Job|A job is a basic construct used to specify the repository and output used in a migration or index. This is how you connect two systems together with Federation Services.| |Mapping|Provides metadata mapping between types/aspects from a source system to an output system. These can be used in a job and in a content services connector.| diff --git a/federation-services/latest/install/index.md b/federation-services/latest/install/index.md index b783329e80..3b8c584fb1 100644 --- a/federation-services/latest/install/index.md +++ b/federation-services/latest/install/index.md @@ -8,22 +8,24 @@ The Federation Services capability for Alfresco Content Services is delivered in Check the [supported platforms]({% link federation-services/latest/support/index.md %}) for information on what you require before you start the installation. -> **Note:** A compatible version of Alfresco Governance Services (if you plan to use the Manage in Place capabilities) is required, for example: if using Alfresco Content Services 23.1, make sure that you install Alfresco Governance Services 23.1. +> **Note:** A compatible version of Alfresco Governance Services is required if you plan to use the Manage in Place capabilities. For example, if you're using Alfresco Content Services 23.1, make sure that you install Alfresco Governance Services 23.1. You can download the Federation Services software from [Hyland Community](https://community.hyland.com/){:target="_blank"}. ### Software requirements -* Java JRE 17+ -* Tomcat 9 - * Memory pool: 4GB required, 8GB recommended. This can be updated in the Java tab of your Apache Tomcat Properties window. +See [Supported platforms]({% link federation-services/latest/support/index.md %}) for required versions. - > **Note:** We recommend using a separate instance, where possible, instead of using the same one used by Alfresco Content Services. +* Java: Java JRE is required +* Application server: Apache Tomcat + * Memory pool: 4GB required, 8GB recommended. This can be updated in the Java tab of your Apache Tomcat **Properties** window. + + > **Note:** A separate instance is recommended, where possible, instead of using the same one used by Alfresco Content Services. > **Note:** Federation Services runs on the Spring platform, which does not support Tomcat versions beyond 9. -* MongoDB 6.x -* Linux or Windows +* Database: MongoDB +* Operating systems: Linux or Windows #### Client Access @@ -54,39 +56,55 @@ Federation Services performance is based on a number of factors: Federation Services uses a database for configuration, job state, and auditing data. Auditing data can also be configured to go to a log file. -Finally, Federation Services will be integrating 2 or more products. Performance will be determined by the above factors, but also by the performance of each system being integrated. Note: Federation Services is an I/O intensive application and therefore should be provisioned accordingly. Recommend SSD or NVMe based back end storage with enough capacity to index the entire contents of the source data. +Finally, Federation Services will be integrating 2 or more products. Performance will be determined by the above factors, but also by the performance of each system being integrated. + +> **Note:** Federation Services is an I/O intensive application and therefore should be provisioned accordingly. Recommend SSD or NVMe based back end storage with enough capacity to index the entire contents of the source data. ## Install steps These steps describe how to install Federation Services to an instance of Alfresco Content Services. -All files necessary for install are available from [Hyland Community](https://community.hyland.com/){:target="_blank"}. Sign in, select **Support > Alfresco Downloads**, and search for the product you require. + +You can download the release files from [Hyland Community](https://community.hyland.com/){:target="_blank"}. Log in, select the **Support** tab, and then the **Alfresco Downloads** option under **Software Downloads**. Search for the required product version and navigate to the version page. 1. Download the files provided for the Federation Services release. - This should include the following: + This should include the following: - * `AFS-federation.war` - * `AFS-admin.war` + * `AFS-federation.war` + * `AFS-admin.war` -2. Copy the Federation Services Admin and discovery WAR files or expanded zip to **Tomcat Installation** `Directory/webapps` directory. +2. Copy the Federation Services Admin and discovery WAR files or expanded ZIP to the Tomcat installation directory, `/Tomcat/webapps`. 3. Start Tomcat. -4. Navigate to the `mongo-db.properties` file `Tomcat/Webapps/3Sixty/Admin/WEB-INF/classes/mongo-db.properties`. +4. Navigate to the `mongo-db.properties` file, `Tomcat/webapps/3Sixty/Admin/WEB-INF/classes/mongo-db.properties`. 5. Set the credentials for the user and the database connection details. -6. Depending on what database you created the admin user in, you may have to append the database name to the end of the uri. This is the authenticating database. The following example config assumes you followed the previous steps exactly. + + Depending on what database you created the admin user in, you may have to append the database name to the end of the URI. This is the authenticating database. + + The following example configuration assumes you followed the previous steps exactly: + + ```text + mongo.db.username=simflofy + mongo.db.password=#whatever password you set during mongo user creation + mongo.db.uri=mongodb://[[USER]]:[[PASS]]@localhost:27017/simflofy + ``` + +6. In the same folder, open `simflofy-global.properties` and check if all of the 'initialize' properties are set to `true`. If not, set them to `true` and save the file. + + See the section on [properties]({% link federation-services/latest/admin/admin-properties.md %}) for more information. + +7. If you have a shared loader enabled in Tomcat, move the global properties files to the `shared/classes` folder. Now the properties files will persist after redeploying the WAR. +8. Restart Tomcat. +9. In `simflofy-global.properties` set initialize properties to `false`. +10. Copy and paste the following into your browser: ```text - mongo.db.username=simflofy - mongo.db.password=#whatever password you set during mongo user creation - mongo.db.uri=mongodb://[[USER]]:[[PASS]]@localhost:27017/simflofy + http://(servername):(port)/3sixty-admin ``` -7. In the same folder, open `simflofy-global.properties` and check if all of the 'initialize' properties are set to true. If not, set them to `true` and save. See the section on properties for more information. -8. If you have a shared loader enabled in Tomcat, move the global properties files to the `shared/classes` folder. Now the properties files will persist after redeploying the WAR. -9. Restart Tomcat. -10. In `simflofy-global.properties` set initialize properties to false. -11. Copy and paste the following into your browser: `http://(servername):(port)/3sixty-admin`. The default Federation Services Username/Password is admin/admin. -12. Access the Admin app through your preferred browser to [configure]({% link federation-services/latest/config/index.md %}) your installation. + The default Federation Services username/password is `admin/admin`. + +11. Access the Admin app through your preferred browser to [configure]({% link federation-services/latest/config/index.md %}) your installation. ## Configure Tomcat for SSL -In order to finish the installation of Federation Services, you need to generate or install a certificate and enable SSL and and Port 8443 or port 443 in `tomcat/conf/server.xml`. Go to [Tomcat SSL documentation](https://tomcat.apache.org/tomcat-9.0-doc/ssl-howto.html){:target="_blank"} for more information. +To finish the installation of Federation Services, you need to generate or install a certificate, enable SSL, and port 8443 or port 443 in `tomcat/conf/server.xml`. Go to the [Tomcat SSL documentation](https://tomcat.apache.org/tomcat-9.0-doc/ssl-howto.html){:target="_blank"} for more information. diff --git a/federation-services/latest/support/index.md b/federation-services/latest/support/index.md index 702b019516..66e9128e22 100644 --- a/federation-services/latest/support/index.md +++ b/federation-services/latest/support/index.md @@ -7,10 +7,15 @@ The following are the supported platforms for Alfresco Federation Services: | Version | Notes | | ------- | ----- | | Content Services 23.x | | -| Content Services 7.4 | | | | | | **Java** | | -| Java JRE 17 or later | | +| Java JRE 17 | | +| | | +| **Databases** | | +| MongoDB 6 | | +| | | +| **Application servers** | | +| Tomcat 9 | | | | | | **Browsers** | | | Chrome (latest) | | diff --git a/federation-services/latest/using/index.md b/federation-services/latest/using/index.md index aa6cb55e2b..2e160174b6 100644 --- a/federation-services/latest/using/index.md +++ b/federation-services/latest/using/index.md @@ -2,17 +2,20 @@ title: Using Federation Services --- +Once your Federation Services environment has been successfully installed, you are ready to start making connections and executing jobs. + ## Quick Start User Guide -Once your Federation Services environment has been successfully installed, you are ready to start making connections and executing jobs. This guide will walk you through this process. For this example, we will be using Google Drive as our source (repository) location and Dropbox as our target location (output). -**Step 1.** Login to your Federation Services environment once all the required components have been installed. -**Step 2.** Create the repository connection for your source data that you want to transfer. -**Step 3.** Create your output connection for your target location you want to move your data to. -**Step 4.** Create a new integration job to sync your two new connections. -**Step 5.** Create a task to filter your results before moving your data. -**Step 6.** Map your fields to tell Federation Services where you want the data in each field moved to. -**Step 7.** Run your new job to begin the data transfer process. -**Step 8.** View your results to confirm the transfer was successful. +This guide will walk you through this process. For this example, we will be using Google Drive as our source (repository) location and Dropbox as our target location (output). + +* **Step 1.** Login to your Federation Services environment once all the required components have been installed. +* **Step 2.** Create the repository connection for your source data that you want to transfer. +* **Step 3.** Create your output connection for your target location you want to move your data to. +* **Step 4.** Create a new integration job to sync your two new connections. +* **Step 5.** Create a task to filter your results before moving your data. +* **Step 6.** Map your fields to tell Federation Services where you want the data in each field moved to. +* **Step 7.** Run your new job to begin the data transfer process. +* **Step 8.** View your results to confirm the transfer was successful. **Step 1. Login to your Federation Services environment** @@ -123,7 +126,7 @@ NEXT STEP: View your results 1. Once the integration job is complete, you can view your results. This shows that all the files have been successfully integrated from one system to the other. If you log into your target repository you will see these files have been added successfully. -**Congratulations! You have successfully created your first Integration Job.** +Congratulations! You have successfully created your first Integration Job. ## Discovery and Analytics @@ -297,13 +300,13 @@ Content View Connectors are used by Content Views in Federation to query indexed **Connector ID** Connector ids are how Federation Services identifies the individual connector when receiving calls from other sources, such as Federation Service. This value must be usable as part of url. Use the description field if you need more than a few letters/numbers to describe the connection. The description shows up with its connectorId across the product. -* **Connector ID:** A unique identifier for this connection i.e. simflofy_demo (Alphanumeric, dashes and underscore characters only) +* **Connector ID:** A unique identifier for this connection i.e. `simflofy_demo` (alphanumeric, dashes and underscore characters only). * **Description:** The text that will be displayed on drop-downs etc. to identify this connection. * **Type:** The type of Search Connection (Solr, Mongo, Elastic etc.) * **Keep Connection Alive:** Federation Services will cache the connection for a given amount of time before discarding it. -* **Keep Alive in Milliseconds:** How long to keep the connection alive before discarding it (300000 is 5 minutes) +* **Keep Alive in Milliseconds:** How long to keep the connection alive before discarding it (`300000` is 5 minutes). * **Security Mode:** This is how to authenticate with the back-end search. -* **Authentication Connection:** The most common method is to use the appropriate authentication connection +* **Authentication Connection:** The most common method is to use the appropriate authentication connection. * **User Pass-through Credentials:** Users the authenticates with whatever authentication they used for Federation Services. Only supported in rare cases. ##### Result Links @@ -315,7 +318,7 @@ If **Download** is selected, the file names in your view will call a document do If **External** is selected, you will need to add **Result Links**. When you click the Result Links button a modal should appear. It takes three arguments * **Content Service Connector:** External link configurations are grouped by content service connector. - * This allows documents from different repositories to form different links + * This allows documents from different repositories to form different links. * **Link Field:** The document field that contains relevant information for building the link. * **Link Url:** The content of the link field will be appended to this url to create the link. @@ -342,7 +345,7 @@ Individuals connectors might have specific fields here that were not general eno Additionally, this is where you can use the "Add Custom Parameter" button to set any default query values for the connection. -A query_fq configuration param lets you define facet queries behind the scene. This is done to provide limited views or subsets of data in the search. Essentially you could create any number of views on the same date but each view would display different results. This can also be used in a role base system where you have views setup for specific user roles.Unless the fq is already encoded, you will need to wrap it in the encode() function where it will be URLEncoded UTF-8. +A `query_fq` configuration parameter lets you define facet queries behind the scene. This is done to provide limited views or subsets of data in the search. Essentially you could create any number of views on the same date but each view would display different results. This can also be used in a role base system where you have views setup for specific user roles.Unless the `fq` is already encoded, you will need to wrap it in the `encode()` function where it will be URLEncoded UTF-8. The syntax is: @@ -384,11 +387,11 @@ Federation Services's Content Service Connections offer public REST endpoints th > **Tip:** Connector ids are how Federation Services identifies the individual connector when receiving calls from other sources, such as Federation Service. This value must be usable as part of url. Use the description field if you need more than a few letters/numbers to describe the connection. The description shows up with its connector ID across the product. -* **Connector ID**: A unique identifier for this connection i.e. simflofy_demo (Alphanumeric, dashes and underscore characters only) +* **Connector ID**: A unique identifier for this connection i.e. `simflofy_demo` (alphanumeric, dashes and underscore characters only) * **Description**: The text that will be displayed on drop-downs etc. to identify this connection. * **Type**: The type of Search Connection (Solr, Mongo, Elastic etc.) * **Keep Connection Alive**: Federation Services will cache the connection for a given amount of time before discarding it. -* **Keep Alive in Milliseconds**: How long to keep the connection alive before discarding it (300000 is 5 minutes) +* **Keep Alive in Milliseconds**: How long to keep the connection alive before discarding it (`300000` is 5 minutes) * **Security Mode**: This is how to authenticate with the back-end search. * **Authentication Connection**: The most common method is to use the appropriate authentication connection * **User Pass-through Credentials**: Users the authenticates with whatever authentication they used for Federation Services. Only supported in rare cases. @@ -483,7 +486,9 @@ You will be brought to the generic Content Connection page. ##### GET FILE -`GET /api/repo//file?id=` +```text +GET /api/repo//file?id= +``` **Description**: @@ -499,11 +504,15 @@ You will be brought to the generic Content Connection page. **With CURL** -`curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/file?id=384896487495" | json_pp` +```bash +curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/file?id=384896487495" | json_pp +``` ##### GET OBJECT ID BY PATH -`GET /api/repo//idbypath?fileName=&folderPath=` +```text +GET /api/repo//idbypath?fileName=&folderPath= +``` **Description**: @@ -531,11 +540,15 @@ You will be brought to the generic Content Connection page. **With CURL** -`curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/idbypath?fileName=testFolder&folderPath=/TestFolder/ | json_pp` +```bash +curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/idbypath?fileName=testFolder&folderPath=/TestFolder/ | json_pp +``` ##### POST FILE -`POST /api/repo//file?fileName=&folderId=&type=` +```text +POST /api/repo//file?fileName=&folderId=&type= +``` **Description**: @@ -568,8 +581,10 @@ You will be brought to the generic Content Connection page. **With CURL** -`curl -u admin:admin -Ffile=@/Users/simflofy/CaterpillarDEUAjax.pdf"` -`localhost:8081/3sixty-admin/api/repo/box/file?fileName=Caterpillar%20DEU%20Ajax.pdf&folderId=105965269305&type=document" | json_pp` +```bash +curl -u admin:admin -Ffile=@/Users/simflofy/CaterpillarDEUAjax.pdf" +localhost:8081/3sixty-admin/api/repo/box/file?fileName=Caterpillar%20DEU%20Ajax.pdf&folderId=105965269305&type=document" | json_pp +``` ##### UPDATE FILE @@ -607,7 +622,7 @@ fileId=&folderId=&fileName=&type=&property1=/updateContent?fileId=` +```text +PUT /api/repo//updateContent?fileId= +``` **Description**: @@ -641,7 +658,7 @@ localhost:8081/3sixty-admin/api/repo/box/update?fileName=Caterpillar%20DEU%20Aja **With CURL** -```text +```bash curl -X PUT -u admin:admin --data-binary '@/Users/simflofy/CaterpillarDEUAjax-newversion.pdf' 'localhost:8081/3sixty-admin/api/repo/box/updateContent?fileId=804939960448' | json_pp @@ -649,7 +666,9 @@ curl -X PUT -u admin:admin --data-binary '@/Users/simflofy/CaterpillarDEUAjax-ne ##### UPDATE FILE PROPERTIES -`PUT /api/repo//updateProperties?fileId=&=&=` +```text +PUT /api/repo//updateProperties?fileId=&=&= +``` **Description**: @@ -678,7 +697,7 @@ curl -X PUT -u admin:admin --data-binary '@/Users/simflofy/CaterpillarDEUAjax-ne **With CURL** -```text +```bash curl -X PUT -u admin:admin 'localhost:8081/3sixty-admin/api/repo/box/updateProperties?fileId=804939960448&textfield=newvalue' | json_pp @@ -716,7 +735,7 @@ curl -X PUT -u admin:admin **With CURL** -```text +```bash curl -u admin:admin -X POST "localhost:8081/3sixty-admin/api/repo/cmis/checkin? id=5dba1525-44a6-45ed-a42e-4a155a3f0539&comment=Spacing%20Fix" | json_pp ``` @@ -752,14 +771,16 @@ id=5dba1525-44a6-45ed-a42e-4a155a3f0539&comment=Spacing%20Fix" | json_pp **With CURL** -```text +```bash curl -u admin:admin -X POST "localhost:8081/3sixty-admin/api/repo/cmis/checkout? id=5dba1525-44a6-45ed-a42e-4a155a3f0539" | json_pp ``` ##### GET FILE BATCH (3.1.1 and older) -`GET /api/repo/__BATCH__/file?id[x]=&conn[x]=&fname[x]=` +```text +GET /api/repo/__BATCH__/file?id[x]=&conn[x]=&fname[x]= +``` **Description**: @@ -767,7 +788,7 @@ id=5dba1525-44a6-45ed-a42e-4a155a3f0539" | json_pp **Query Parameters**: -**Note:** [X] EQUALS AN INTEGER +> **Note:** [X] EQUALS AN INTEGER * **idx**: ID of the file to retrieve * **connx**: Connector id of the file @@ -780,7 +801,7 @@ id0=/home/user/simflofy/test.txt&conn0=localFS&fname0=test.txt&id1=629425696136& **With CURL** -```text +```bash curl -u admin:admin -X "localhost:8081/3sixty-admin/api/repo/__BATCH__/file? id0=/home/user/simflofy/test.txt&conn0=localFS&fname0=test.txt&id1=629425696136&conn1=box&fname1=Caterpillar%20DEU%20Ajax.pdf" ``` @@ -835,7 +856,9 @@ connId: "box" ##### GET FILE VERSIONS -`GET /api/repo//listversions?id=` +```text +GET /api/repo//listversions?id= +``` **Description**: @@ -849,7 +872,9 @@ connId: "box" * **id**: Source repository id. Differs depending on repository. -`GET /api/repo/cmis/listversions?id=5dba1525-44a6-45ed-a42e-4a155a3f0539` +```text +GET /api/repo/cmis/listversions?id=5dba1525-44a6-45ed-a42e-4a155a3f0539 +``` **Returns**: @@ -879,7 +904,7 @@ connId: "box" **With CURL** -```text +```bash curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/spo/listversions? id=5dba1525-44a6-45ed-a42e-4a155a3f0539" | json_pp ``` @@ -921,7 +946,7 @@ id=5dba1525-44a6-45ed-a42e-4a155a3f0539" | json_pp **With CURL** (remember to encode spaces and quotes) -```text +```bash curl -u admin:admin -X DELETE "localhost:8081/3sixty-admin/api/repo/box/delete? id=629425696136&allversions=false" | json_pp ``` @@ -930,7 +955,9 @@ id=629425696136&allversions=false" | json_pp **Request**: -`GET /api/repo//folderitems?id=` +```text +GET /api/repo//folderitems?id= +``` **Description**: @@ -944,7 +971,9 @@ id=629425696136&allversions=false" | json_pp * **id**: Source repository id. Differs depending on repository. -`GET /api/repo/box/folderitems?id=384896487495` +```text +GET /api/repo/box/folderitems?id=384896487495 +``` **Returns**: @@ -1021,7 +1050,9 @@ id=629425696136&allversions=false" | json_pp **With CURL** -`curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/folderitems?id=384896487495" | json_pp` +```bash +curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/folderitems?id=384896487495" | json_pp +``` ##### CREATE FOLDER @@ -1054,11 +1085,15 @@ id=629425696136&allversions=false" | json_pp **With CURL** -`curl -u admin:admin -X POST"localhost:8081/3sixty-admin/api/repo/box/folder?path=3Sixty/testfolder" | json_pp` +```bash +curl -u admin:admin -X POST"localhost:8081/3sixty-admin/api/repo/box/folder?path=3Sixty/testfolder" | json_pp +``` ##### GET ITEM PROPERTIES -`GET /api/repo//properties?id=` +```text +GET /api/repo//properties?id= +``` **Description**: @@ -1105,7 +1140,9 @@ id=629425696136&allversions=false" | json_pp **With CURL** -`curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/properties?id=384896487495" | json_pp` +```bash +curl -u admin:admin -X GET "localhost:8081/3sixty-admin/api/repo/box/properties?id=384896487495" | json_pp +``` ### Discovery @@ -1129,7 +1166,7 @@ Many of these can be accessed by Discovery Admins at `tsearch/config`. It is ava #### Widget Instances - Widget Instance is an instance of a Widget Definition that can now be placed in a Content View. Many instances of the same definition can exist in a content view. Each instance has a unique id. Federation Services comes preloaded with a number of common widget instances. If you don't find the type you need in your list, found under **Federation > Widget Instances**, you will need to create one. +Widget Instance is an instance of a Widget Definition that can now be placed in a Content View. Many instances of the same definition can exist in a content view. Each instance has a unique id. Federation Services comes preloaded with a number of common widget instances. If you don't find the type you need in your list, found under **Federation > Widget Instances**, you will need to create one. In the Widget Instance list you can search and sort by name, widget ID, label and field. @@ -1275,7 +1312,7 @@ Event jobs, in abstract are meant to be triggered by some external action. For a Event Configurations enhance Event Jobs and require: -* simflofy.event.queue=true (which is the default setting) +* `simflofy.event.queue=true` (which is the default setting) It allows them to be triggered through content service actions. So, if we want to add to that tutorial we would need to add a scenario in which we: @@ -1357,7 +1394,9 @@ We will now call the API to push the document into the queue for processing. URL format: -`http://{HOST}:{PORT}/3sixty-admin/api/event/service/pushevent?jobId=1631569493226&documentId=C:/SourceDocuments/sampledoc.pdf` +```text +http://{HOST}:{PORT}/3sixty-admin/api/event/service/pushevent?jobId=1631569493226&documentId=C:/SourceDocuments/sampledoc.pdf +``` Example curl command: @@ -1745,7 +1784,7 @@ Federation Services allows users to schedule integration jobs and job groups to 2. Select the Create New Job Schedule button at the top of the page. 3. Give your scheduled job a **name**. 4. Fill out the Configuration Fields (descriptions below). -5. Click the Update Configuration button once you are done configuring the scheduled job for it to start running at the set time and interval. +5. Click the **Update Configuration** button once you are done configuring the scheduled job for it to start running at the set time and interval. > **Note:** If the schedule on a running job changes, that job will be aborted and the new schedule will take effect immediately. @@ -1765,14 +1804,20 @@ Federation Services allows users to schedule integration jobs and job groups to At the top of the scheduled job you can see the current status. -* **Current Status**: Unsaved until Update Configuration button is pressed. Then OFF, WAITING, or RUNNING. -* **Time until next run**: As it reads. If this runs to zero, it will not refresh unless the schedule is closed and reopened. Will Display as NA if performing a single run (interval set to 0) or the job is currently running. +* **Current Status**: `Unsaved` until **Update Configuration** button is pressed. Then `OFF`, `WAITING`, or `RUNNING`. +* **Time until next run**: As it reads. If this runs to zero, it will not refresh unless the schedule is closed and reopened. Will Display as `NA` if performing a single run (interval set to `0`) or the job is currently running. #### List Jobs A Federation Services Job is the process of moving or syncing content(including versions, ACL's, metadata) from one CMS (content management system) to another. Since Federation Services Jobs are specifically engineered for content management systems, moving content and metadata is just point and click. There are many Job types as well as Job Tasks that can handle anything from data validation and cleansing to duplication detection. -The List jobs page shows the jobs created. There are five available Job Types: Sync, Simple Migration, Incremental Migration, Event and Polling. +The List jobs page shows the jobs created. The available Job Types are: + +* Sync +* Simple Migration +* Incremental Migration +* Event +* Polling ##### Job Types @@ -1793,7 +1838,7 @@ To access Job Groups, go to the group section in the left sidebar and select Job ###### Creating a Job Group -To access Job Groups, go to the group section in the left sidebar and select Job Groups. From here you can view and create Job Groups. +To access Job Groups, go to the group section in the left sidebar and select **Job Groups**. From here you can view and create Job Groups. 1. Click **Create New Job Group**. 2. Fill in the **group's name**. @@ -1810,7 +1855,7 @@ To access Job Groups, go to the group section in the left sidebar and select Job * **Repository Connection**: The Repository Connection used for this Job. * **Output Connection**: The Output Connection used for this Job. * **Type**: The type of Federation Services Job. -* **Job Tags**: Another option for separating jobs into groups. If any jobs have tags, they will appear as a filtering option when listing jobs. Tags are case sensitive. +* **Job Tags**: Another option for separating jobs into groups. If any jobs have tags, they will appear as a filtering option when listing jobs. Tags are case sensitive. * **Include documents modified after this date and time**: Referred to as "Start Time", this time will be used to check the last modified date of files before processing them. * **Include documents modified before this date and time**: Referred to as "End Time", this time will be used to check the last modified date of files before processing them. @@ -1840,7 +1885,7 @@ Notifications will only function is email is enabled. This allows the user to se * **Repository Connection Thread Count**: The number of individual workers which will process and queue documents. Implementation may vary in certain systems. * **Output Connection Thread Count**: The number of individual workers which will process and post documents. Implementation may vary in certain systems. -* **Batch Size**: If greater than 0, Documents will be assigned a batch ID. Batch Ids are a combination of the job id and the job run id (the timestamp of when the job was started), plus the batch number. The batchId will also be set as the highest parent folder for the document. (For example, /home/simflofy will become {batchId}/home/simflofy). Some connectors have their own batch configuration, which is compatible with this field. +* **Batch Size**: If greater than 0, Documents will be assigned a batch ID. Batch Ids are a combination of the job id and the job run id (the timestamp of when the job was started), plus the batch number. The batchId will also be set as the highest parent folder for the document. (For example, `/home/simflofy` will become `{batchId}/home/simflofy`). Some connectors have their own batch configuration, which is compatible with this field. * **Max Queue Size**: Limits the number of documents that can be queued at once, making repository workers wait. This can slow down output speeds, which may be useful if your output repository can be throttled. * **Max Errors Allowed**: Maximum errors before the job stops. Leave 0 for unlimited errors. * **Process Relationships** : Check this to tell Target Connector to process Document Relationships. This feature only works for the CMIS Connector. @@ -1854,15 +1899,15 @@ Notifications will only function is email is enabled. This allows the user to se Job tasks provide a processing pipeline for documents and metadata. This means doing things like: -1. Filtering content out or in based on rules, such as metadata values -2. Cleaning up file names or file paths -3. Re-Parenting based on rules -4. Extracting metadata from paths -5. Adding metadata from third parties such as Databases, Rest APIs, OCR Engines, etc... +1. Filtering content out or in based on rules, such as metadata values. +2. Cleaning up file names or file paths. +3. Re-Parenting based on rules. +4. Extracting metadata from paths. +5. Adding metadata from third parties such as Databases, Rest APIs, OCR Engines, etc.. 6. Transforming documents from one mime type to another, such as Word to PDF. -7. Transforming metadata -8. PII - Personally Identifiable Information - detection -9. De-Duplication +7. Transforming metadata. +8. PII - Personally Identifiable Information - detection. +9. De-Duplication. ##### Adding Tasks to Jobs @@ -1931,7 +1976,7 @@ A temporary job is created using the repository configuration of the selected jo ##### Running the PII Scan -Under Analytics select PII Scan and complete the PII Scan page +Under Analytics select PII Scan and complete the PII Scan page: * Select the repository connection you want to scan * Select the Job to scan for PII @@ -1944,10 +1989,10 @@ Under Analytics select PII Scan and complete the PII Scan page ##### Viewing PII Data -Once the PII is ran users can view the PII report in the Discovery UI by either: +Once the PII scan runs, users can view the PII report in the Discovery UI by either: -* Clicking on the link provided in the Scan complete notification box or -* Selecting **PII Data Viewer** in the left navigation pane on the home page of the Discovery UI +* Clicking on the link provided in the Scan complete notification box. +* Selecting **PII Data Viewer** in the left navigation pane on the home page of the Discovery UI. #### Document Duplication Detection and Reporting @@ -1961,13 +2006,17 @@ After crawling your source system and outputting to the Federation Services Repo Depending on how many documents you found during the crawl > 100,000, you may need to add docHash index. -`db.tsRecordProcessed.createIndex( { docHash: 1 } )` +```text +db.tsRecordProcessed.createIndex( { docHash: 1 } ) +``` -Next we group by docHash and output to a new collection named duplicates (You can name the new output collection to anything you like). +Next we group by docHash and output to a new collection named duplicates (you can name the new output collection anything you like): -`db.tsRecordProcessed.aggregate([{$group:{_id:"$docHash",docs:{$push:"$doc_id"},doc_names:{$push:"$doc_name"}}},{$project:{docs:1,doc_names:1,numDocs:{$size:"$docs"}}},{$match:{numDocs:{$gt:1}}},{$out:"duplicates"}])` +```text +db.tsRecordProcessed.aggregate([{$group:{_id:"$docHash",docs:{$push:"$doc_id"},doc_names:{$push:"$doc_name"}}},{$project:{docs:1,doc_names:1,numDocs:{$size:"$docs"}}},{$match:{numDocs:{$gt:1}}},{$out:"duplicates"}]) +``` -You can now export de-dupes collection to CSV or JSON using mongoexport +You can now export de-dupes collection to CSV or JSON using `mongoexport`: ```text mongoexport --db simflofy --collection duplicates --fields _id,docs,doc_names @@ -1976,8 +2025,7 @@ mongoexport --db simflofy --collection duplicates --fields _id,docs,doc_names #### Stuck Jobs -If a job is running and the **Abort** does not stop the job in an appropriate amount of time, the job can be killed manually through an Admin page. -See [Active Jobs]({% link federation-services/latest/admin/index.md%}#active-jobs) for more information. +If a job is running and the **Abort** does not stop the job in an appropriate amount of time, the job can be killed manually through an Admin page. See [Active Jobs]({% link federation-services/latest/admin/index.md%}#active-jobs) for more information. ### Federation Services Job Flow @@ -1991,9 +2039,9 @@ Tasks are run for each document that has been read. A number of tasks can be set #### Duplication Detection -This task checks the chosen field to find duplicate documents during the job run. And will take the selected actions against the document if a duplicate one exists. To see how each repository handles duplicates and versioning check the individual connector page. +This task checks the chosen field to find duplicate documents during the job run. It takes the selected actions against the document if a duplicate one exists. To see how each repository handles duplicates and versioning check the individual connector page. -When the duplication check is ran multiple times the original file will not be marked as duplicate. +When the duplication check is run multiple times, the original file will not be marked as duplicate. ##### Configuration @@ -2813,9 +2861,9 @@ Extracts ACLs from the Windows or Linux filesystem document and adds them to the This task will have some different behaviour depending on your operating system. In a POSIX environment (macOS or Linux) permissions may be added as the field `document.permissions` with the permissions in a semicolon(;) delimited list, if any exist. -If the filesystem supplies an owner, it will be added as **simflofy.owner** +If the filesystem supplies an owner, it will be added as `simflofy.owner`. -Additionally, simflofy will create a permission map of the principals and their permissions. It will set is as the **originalPermissions** field, so +Additionally, simflofy will create a permission map of the principals and their permissions. It will set is as the **originalPermissions** field: ```text Map> permissions = new Map<>(); @@ -2823,15 +2871,13 @@ Map> permissions = new Map<>(); rd.setOriginalPermissions(permissions); ``` -Finally, if any User Defined File Attributes (extended attributes), they will be added as a semicolon delimited list in the field - -`simflofy.userattributes` +Finally, if any User Defined File Attributes (extended attributes), they will be added as a semicolon delimited list in the field `simflofy.userattributes`. #### Generic ACL Mapper The generic ACL mapper job task allows you to create simple rules for matching principles and permissions from one system to another. ACLs will need to be extracted from each document. This task reads the **originalPermissions** field of the document and sets the **transformedPermissions** field. -LIMITED USAGE +LIMITED USAGE Only the Azure Blob, CMIS, and Alfresco Connectors can use this task. For all other acl mapping, a [JavaScript](#javascript-processing) task is required. * Process Files and Process Folders tells the task what to process.