diff --git a/docs/bist_smoke_tests.md b/docs/bist_smoke_tests.md
index bbeaa875..9d0fdd67 100644
--- a/docs/bist_smoke_tests.md
+++ b/docs/bist_smoke_tests.md
@@ -41,7 +41,7 @@ It is recommended to run **BIST** on a new workspace or on a test workspace as
1. Navigate to Library Application (Admin), locate Trace and install it
into Test Workspace
- 2. Ensure that Trace Agent and all dependent Agents are enabled and working
+ 2. Ensure that Trace Agent and all dependent Agents are enabled and working
3. Within the workspace, navigate to Trace → Setup and set Run Option →
Continuous
@@ -68,7 +68,7 @@ It is recommended to run **BIST** on a new workspace or on a test workspace as
4. Generate test data emails with attachments
- 5. Extract and Ingest 143 documents with various file types, language
+ 5. Extract and Ingest 144 documents with various file types, language
contents across 2 data batches
4. Create Terms **[AUTOMATED]**
@@ -132,7 +132,7 @@ It is recommended to run **BIST** on a new workspace or on a test workspace as
- Trace BIST Data Source Generated Batches: `2`
-- Total Documents in Trace BIST folder: `142`
+- Total Documents in Trace BIST folder: `143`
- Total Deleted Documents: `1`
diff --git a/docs/demo_guide.md b/docs/demo_guide.md
index ecfd4cce..41ddf8d4 100644
--- a/docs/demo_guide.md
+++ b/docs/demo_guide.md
@@ -87,35 +87,36 @@ DEMO A: Enabling Office 365 Data Source and auto-tagging documents (alerts)
2. Click into the **Trace:Rules** tab to see sample rules in the workspace.
Rules are how you define the type of content that is relevant to you. The
**Market Abuse** rule has been pre-created for you along with a few others.
- 
+
+
+
+3. Click into the **Market Abuse** rule and see the options that make up this
+ rule:
-3. Click into the **Market Abuse** rule and see the options that make up this
- rule:
+ - **Searchable Set:** A Saved Search that the rule runs against
- - **Searchable Set:** A Saved Search that the rule runs against
+ - **Associated Actions:** A set of actions to take on matched documents (this
+ rule will move the matched documents to a separate folder)
- - **Associated Actions:** A set of actions to take on matched documents (this
- rule will Tag and Batch out to reviewers)
+ - **Terms**: Allows you to link Trace Terms to refine your document filter
+ criteria with an e-discovery grade searching engine (dtSearch)
- - **Terms**: Allows you to link Trace Terms to refine your document filter
- criteria with an e-discovery grade searching engine (dtSearch)
+ - The rule is currently **Disabled**
+ 
- - The rule is currently **Disabled**
- 
+4. Click the **Enable Rule** console button under **Trace Rule Management** section:
-4. Click the **Enable Rule** console button under **Trace Rule Management** section:
+ - The rule is now **Enabled**
- - The rule is now **Enabled**
-
- - As data is ingested, this rule will be automatically applied to all incoming
- documents
- 
+ - As data is ingested, this rule will be automatically applied to all incoming
+ documents
+ 
### Section 2: Importing Data and Running Rules
1. Now let’s test our rules by adding live stream of data into the workspace
with Office 365 Data Source. Go to the **Trace:Data Sources** tab
- 
+ 
2. Click on the **Office 365 Exchange** data source to see how it is configured
@@ -132,69 +133,68 @@ DEMO A: Enabling Office 365 Data Source and auto-tagging documents (alerts)
- **Trace Monitored Individuals:** lets you select which people to monitor
- The Data Source is currently **Disabled**
- 
+ 
3. Click **Enable Data Source** on the right hand side
- 
+ 
-4. Wait for Data Batches with data to start showing up – refresh the page
- several times, until you start seeing Data Batches on the bottom (it may
- take several minutes for first data batch to be fully created). You can see
- how much data each Data Batch retrieved and corresponding statuses. Once you
- have a few Data Batches in “Completed” state the rest of the Trace workflow
- automatically triggers.
- 
+4. Data will begin to get pulled from the exchange server for each of the defined Monitored Individuals. Refreshing the page will show data batches as they are pulled from the data source and processed (it may take several minutes for first data batch to be fully created). You can see how much data each Data Batch retrieved and corresponding statuses. Once you have a few Data Batches in “Completed” state the rest of the Trace workflow automatically triggers.
+
+ 
As time goes on, more batches with data will be created and ingested:
### Section 3: Viewing Results
-1. Go to the **Review Batches** tab. This shows all Review Batches that were
- auto-generated from the enabled rules
- 
-
-2. Click on **Documents** tab:
+1. Click on **Documents** tab:
- 1. Data is proactively being retrieved, extracted and ingested
+ 1. Data is proactively being retrieved, extracted and ingested
- 2. “**Trace Search Index**” is built automatically -\> for your ad-hoc
- searching
- 
+ 2. For this demo, all ingested documents are routed to the **Ingestion Folder**
-3. Explore Dashboards
- 
+ 
- 1. Once all the data is ingested and analyzed about 7.6% of ingested documents have
- matched the rules:
- 
+ 3. Documents matched to the **Market Abuse** rule will have the **Default Move To Folder** action execute. The matched documents will appear under the **Alerted Documents** folder.
- 2. All of the Rules that matched:
-
+ 
- 3. File type breakdown of ingested documents:
- 
+ 4. Navigate to the **Trace Demo** root folder. You will find the “**Trace Search Index**” is built automatically for your ad-hoc searching.
+ 
-4. Review matched documents
+3. Explore Dashboards
+ 
- 1. Click on “Market Abuse” on the Rules widget
- 
+ 1. Once all the data is ingested and analyzed about 8.5% of ingested documents have
+ matched the rules:
+ 
- 2. Click on “Revenue Sharing” on the Terms widget
- 
+ **(blank)** denotes the number of documents where no Rules were matched.
+
+2. All of the Rules that matched:
+
+
+3. File type breakdown of ingested documents:
+ 
+
+3. Review matched documents
- 3. Click on one of the documents in the list:
- 
+ 1. Click on “Market Abuse” bar on the Rules widget
+ 
+ 2. Click on “Revenue Sharing” on the Terms widget
+ 
+ 3. Click on the top document in the list:
+ 
+ 4. Explore document details:
+ 
+ 
- 4. Explore document details:
- 
- 
+ - Matched terms are automatically highlighted in the document
-1. Matched terms are automatically highlighted in the document
+ - Document text and meta-data was automatically extracted
-2. Document text and meta-data was automatically extracted
+ - Related documents are displayed in the “Family” section
-3. Related documents are displayed in the “Family” section
### Section 4: Recap
@@ -203,12 +203,12 @@ Thanks for completing Demo A! Quick recap, here’s what we did:
1. Enabled **Market Abuse** rule for continuous evaluation
2. Viewed rules that are based on Saved Searches and Trace Terms that have
- Tagging and Batching as associated actions
+ an associated action of moving the document to another folder.
3. Enabled ingestion of documents, proactively from live Office 365 instance
4. Validated that the documents were ingested into the workspace, automatically
- indexed, searched, tagged and batched out for review
+ indexed, searched, tagged and moved
5. Reviewed a matched document in the viewer with extracted data, related
documents and highlighted terms
@@ -225,13 +225,14 @@ DEMO B: Running an Archive Rule
Search** browser in the **Documents** tab and select the “**Non-alerted
Documents**” Saved Search. You’ll notice there are \~5k documents that are
currently in this Saved Search.
- 
- > **NOTE:** Count on non-alerted documents might vary
-
+ 
+
+> **NOTE:** Count on non-alerted documents might vary
+
3. Click into the **Trace:Rules** tab to see all rules in the workspace. The
**Delete Rule** has been pre-created for you. Click into the **Delete
Rule**.
- 
+ 
4. You’ll notice that the **Delete Rule** follows a similar format to the prior
demo. The rule runs on the “Non-alerted Documents” Saved Search and has the
@@ -239,24 +240,26 @@ DEMO B: Running an Archive Rule
all documents in the Saved Search (outside of the set retention policy) once
the rule is enabled. This will enable you to have an **automated retention
policy** in Relativity.
+
> **NOTE:** For this demo retention has be purposely set to 0 hours, which means that all of the `non-alerted` data will be
- delete as soon as the rule runs
-
+delete as soon as the rule runs
+
5. Now let’s enable the rule so Trace can start enforcing your Data Archive
policy. Go to the **Trace:Rules** tab again and click on the “**Delete
Rule**”
- 
+ 
-6. Click on **Enable Rule** in console button under **Automation** section
- 
+6. Click on **Enable Rule** in console button under **Trace Rule Management** section
+ 
7. Go back to the Saved Search Browser in the Documents tab and select the
- “Documents to Delete” Saved Search. Within a few minutes and you’ll start to
+ “Non-alerted Documents” Saved Search. Within a few minutes you’ll start to
see documents getting deleted from this workspace.
- 
+
+
+
+8. Go to **Trace:Rules** , navigate to the Delete Rule and Disable it
+ 
-8. Go to **Trace:Rules** , navigate to the Delete Rule and Disable it
- 
-
-Let us know what you think! Reach out to `trace@relativity.com` with any
+Let us know what you think! Reach out to `support@relativity.com` with any
feedback. Thanks!
diff --git a/docs/media/0b80dfed320b157afa02f114b3fbd7c1.png b/docs/media/0b80dfed320b157afa02f114b3fbd7c1.png
deleted file mode 100644
index 4c8a21b7..00000000
Binary files a/docs/media/0b80dfed320b157afa02f114b3fbd7c1.png and /dev/null differ
diff --git a/docs/media/0c099bc9669ce508b9a1110a66cd15ca.png b/docs/media/0c099bc9669ce508b9a1110a66cd15ca.png
deleted file mode 100644
index a3ebae52..00000000
Binary files a/docs/media/0c099bc9669ce508b9a1110a66cd15ca.png and /dev/null differ
diff --git a/docs/media/2915c8515dde4765959049040c23c5ef.png b/docs/media/2915c8515dde4765959049040c23c5ef.png
deleted file mode 100644
index 06de91e6..00000000
Binary files a/docs/media/2915c8515dde4765959049040c23c5ef.png and /dev/null differ
diff --git a/docs/media/38c5bd780c5ada1f3042f8ac7052d37d.png b/docs/media/38c5bd780c5ada1f3042f8ac7052d37d.png
deleted file mode 100644
index 49f069b4..00000000
Binary files a/docs/media/38c5bd780c5ada1f3042f8ac7052d37d.png and /dev/null differ
diff --git a/docs/media/406ada4e092e1e343bce785bebec7e9c.png b/docs/media/406ada4e092e1e343bce785bebec7e9c.png
deleted file mode 100644
index 6cdb678b..00000000
Binary files a/docs/media/406ada4e092e1e343bce785bebec7e9c.png and /dev/null differ
diff --git a/docs/media/42528d221727ff6d48bb9e844bfb0ffb.png b/docs/media/42528d221727ff6d48bb9e844bfb0ffb.png
deleted file mode 100644
index 9a2987b5..00000000
Binary files a/docs/media/42528d221727ff6d48bb9e844bfb0ffb.png and /dev/null differ
diff --git a/docs/media/56a2973ce72db92e5bbb474fe8e7c216.png b/docs/media/56a2973ce72db92e5bbb474fe8e7c216.png
deleted file mode 100644
index ebc68622..00000000
Binary files a/docs/media/56a2973ce72db92e5bbb474fe8e7c216.png and /dev/null differ
diff --git a/docs/media/57778d5767b262da54b15eff0d505ff5.png b/docs/media/57778d5767b262da54b15eff0d505ff5.png
deleted file mode 100644
index f039be6f..00000000
Binary files a/docs/media/57778d5767b262da54b15eff0d505ff5.png and /dev/null differ
diff --git a/docs/media/59aecb6c0a0e49227506ca5d22afac87.png b/docs/media/59aecb6c0a0e49227506ca5d22afac87.png
deleted file mode 100644
index e6446ef8..00000000
Binary files a/docs/media/59aecb6c0a0e49227506ca5d22afac87.png and /dev/null differ
diff --git a/docs/media/603645b32fe247cfb851e292c7786beb.png b/docs/media/603645b32fe247cfb851e292c7786beb.png
deleted file mode 100644
index befcf09d..00000000
Binary files a/docs/media/603645b32fe247cfb851e292c7786beb.png and /dev/null differ
diff --git a/docs/media/7284f63897ecd8c4c1d154be240a2039.png b/docs/media/7284f63897ecd8c4c1d154be240a2039.png
deleted file mode 100644
index dbe15ec8..00000000
Binary files a/docs/media/7284f63897ecd8c4c1d154be240a2039.png and /dev/null differ
diff --git a/docs/media/7675cfedd1484500512efe27c13ee17f.png b/docs/media/7675cfedd1484500512efe27c13ee17f.png
deleted file mode 100644
index dc686628..00000000
Binary files a/docs/media/7675cfedd1484500512efe27c13ee17f.png and /dev/null differ
diff --git a/docs/media/87545672cde91318af976d0f7705cc9a.png b/docs/media/87545672cde91318af976d0f7705cc9a.png
deleted file mode 100644
index 3c125791..00000000
Binary files a/docs/media/87545672cde91318af976d0f7705cc9a.png and /dev/null differ
diff --git a/docs/media/888c38c3f153691242bd9fb200911bd4.png b/docs/media/888c38c3f153691242bd9fb200911bd4.png
deleted file mode 100644
index 42d5b14e..00000000
Binary files a/docs/media/888c38c3f153691242bd9fb200911bd4.png and /dev/null differ
diff --git a/docs/media/8c040117f5bbed702213148a2aaa0873.png b/docs/media/8c040117f5bbed702213148a2aaa0873.png
deleted file mode 100644
index 693b96c7..00000000
Binary files a/docs/media/8c040117f5bbed702213148a2aaa0873.png and /dev/null differ
diff --git a/docs/media/9121eef9fa793b0003021602cefabbe2.png b/docs/media/9121eef9fa793b0003021602cefabbe2.png
deleted file mode 100644
index 86b93a2d..00000000
Binary files a/docs/media/9121eef9fa793b0003021602cefabbe2.png and /dev/null differ
diff --git a/docs/media/b263a7470437bb562c35e35c0fc113ed.png b/docs/media/b263a7470437bb562c35e35c0fc113ed.png
deleted file mode 100644
index 1db93b73..00000000
Binary files a/docs/media/b263a7470437bb562c35e35c0fc113ed.png and /dev/null differ
diff --git a/docs/media/c8e3e14b0d67dccba13753f24496b38a.png b/docs/media/c8e3e14b0d67dccba13753f24496b38a.png
deleted file mode 100644
index 48752cfe..00000000
Binary files a/docs/media/c8e3e14b0d67dccba13753f24496b38a.png and /dev/null differ
diff --git a/docs/media/cbcdbe80b994aff6e31fad731db5362e.png b/docs/media/cbcdbe80b994aff6e31fad731db5362e.png
deleted file mode 100644
index 00fafd92..00000000
Binary files a/docs/media/cbcdbe80b994aff6e31fad731db5362e.png and /dev/null differ
diff --git a/docs/media/demo_guide/1571430373664.png b/docs/media/demo_guide/1571430373664.png
new file mode 100644
index 00000000..88582d7a
Binary files /dev/null and b/docs/media/demo_guide/1571430373664.png differ
diff --git a/docs/media/demo_guide/1571775617378.png b/docs/media/demo_guide/1571775617378.png
new file mode 100644
index 00000000..b4350c31
Binary files /dev/null and b/docs/media/demo_guide/1571775617378.png differ
diff --git a/docs/media/demo_guide/1571775652697.png b/docs/media/demo_guide/1571775652697.png
new file mode 100644
index 00000000..b4350c31
Binary files /dev/null and b/docs/media/demo_guide/1571775652697.png differ
diff --git a/docs/media/demo_guide/1571775679888.png b/docs/media/demo_guide/1571775679888.png
new file mode 100644
index 00000000..1d7c6a78
Binary files /dev/null and b/docs/media/demo_guide/1571775679888.png differ
diff --git a/docs/media/demo_guide/1571775810589.png b/docs/media/demo_guide/1571775810589.png
new file mode 100644
index 00000000..eaa9c016
Binary files /dev/null and b/docs/media/demo_guide/1571775810589.png differ
diff --git a/docs/media/demo_guide/1571776936298.png b/docs/media/demo_guide/1571776936298.png
new file mode 100644
index 00000000..57407cc8
Binary files /dev/null and b/docs/media/demo_guide/1571776936298.png differ
diff --git a/docs/media/demo_guide/1571777044365.png b/docs/media/demo_guide/1571777044365.png
new file mode 100644
index 00000000..1830c915
Binary files /dev/null and b/docs/media/demo_guide/1571777044365.png differ
diff --git a/docs/media/demo_guide/1571777162502.png b/docs/media/demo_guide/1571777162502.png
new file mode 100644
index 00000000..687a285d
Binary files /dev/null and b/docs/media/demo_guide/1571777162502.png differ
diff --git a/docs/media/demo_guide/1571777552542.png b/docs/media/demo_guide/1571777552542.png
new file mode 100644
index 00000000..5ce7479d
Binary files /dev/null and b/docs/media/demo_guide/1571777552542.png differ
diff --git a/docs/media/demo_guide/1571777717924.png b/docs/media/demo_guide/1571777717924.png
new file mode 100644
index 00000000..1e48527f
Binary files /dev/null and b/docs/media/demo_guide/1571777717924.png differ
diff --git a/docs/media/demo_guide/1571779546803.png b/docs/media/demo_guide/1571779546803.png
new file mode 100644
index 00000000..9992ba14
Binary files /dev/null and b/docs/media/demo_guide/1571779546803.png differ
diff --git a/docs/media/demo_guide/1571779609579.png b/docs/media/demo_guide/1571779609579.png
new file mode 100644
index 00000000..a9761b3e
Binary files /dev/null and b/docs/media/demo_guide/1571779609579.png differ
diff --git a/docs/media/demo_guide/1571779615711.png b/docs/media/demo_guide/1571779615711.png
new file mode 100644
index 00000000..a9761b3e
Binary files /dev/null and b/docs/media/demo_guide/1571779615711.png differ
diff --git a/docs/media/demo_guide/1571779666116.png b/docs/media/demo_guide/1571779666116.png
new file mode 100644
index 00000000..305afc75
Binary files /dev/null and b/docs/media/demo_guide/1571779666116.png differ
diff --git a/docs/media/demo_guide/1571779733791.png b/docs/media/demo_guide/1571779733791.png
new file mode 100644
index 00000000..5383b9e4
Binary files /dev/null and b/docs/media/demo_guide/1571779733791.png differ
diff --git a/docs/media/demo_guide/1571779820919.png b/docs/media/demo_guide/1571779820919.png
new file mode 100644
index 00000000..987a087a
Binary files /dev/null and b/docs/media/demo_guide/1571779820919.png differ
diff --git a/docs/media/demo_guide/Section1.2-1571771958433.png b/docs/media/demo_guide/Section1.2-1571771958433.png
new file mode 100644
index 00000000..8b435c62
Binary files /dev/null and b/docs/media/demo_guide/Section1.2-1571771958433.png differ
diff --git a/docs/media/demo_guide/Section1.2.PNG b/docs/media/demo_guide/Section1.2.PNG
new file mode 100644
index 00000000..1dcf4bb0
Binary files /dev/null and b/docs/media/demo_guide/Section1.2.PNG differ
diff --git a/docs/media/demo_guide/Section1.3.png b/docs/media/demo_guide/Section1.3.png
new file mode 100644
index 00000000..3ed730ac
Binary files /dev/null and b/docs/media/demo_guide/Section1.3.png differ
diff --git a/docs/media/demo_guide/Section1.4.png b/docs/media/demo_guide/Section1.4.png
new file mode 100644
index 00000000..ec851e9f
Binary files /dev/null and b/docs/media/demo_guide/Section1.4.png differ
diff --git a/docs/media/demo_guide/Section2.1.png b/docs/media/demo_guide/Section2.1.png
new file mode 100644
index 00000000..dce352f9
Binary files /dev/null and b/docs/media/demo_guide/Section2.1.png differ
diff --git a/docs/media/demo_guide/Section2.2.png b/docs/media/demo_guide/Section2.2.png
new file mode 100644
index 00000000..76cf6534
Binary files /dev/null and b/docs/media/demo_guide/Section2.2.png differ
diff --git a/docs/media/demo_guide/Section2.2a.png b/docs/media/demo_guide/Section2.2a.png
new file mode 100644
index 00000000..47dc456a
Binary files /dev/null and b/docs/media/demo_guide/Section2.2a.png differ
diff --git a/docs/media/demo_guide/Section2.4.png b/docs/media/demo_guide/Section2.4.png
new file mode 100644
index 00000000..0df22b7e
Binary files /dev/null and b/docs/media/demo_guide/Section2.4.png differ
diff --git a/docs/media/demo_guide/Section3.3.png b/docs/media/demo_guide/Section3.3.png
new file mode 100644
index 00000000..7a6f450c
Binary files /dev/null and b/docs/media/demo_guide/Section3.3.png differ
diff --git a/docs/media/demo_guide/Section3.3i.PNG b/docs/media/demo_guide/Section3.3i.PNG
new file mode 100644
index 00000000..bf76f1f2
Binary files /dev/null and b/docs/media/demo_guide/Section3.3i.PNG differ
diff --git a/docs/media/demo_guide/Section3.3iii.PNG b/docs/media/demo_guide/Section3.3iii.PNG
new file mode 100644
index 00000000..335be638
Binary files /dev/null and b/docs/media/demo_guide/Section3.3iii.PNG differ
diff --git a/docs/media/demo_guide/Section3.4i.PNG b/docs/media/demo_guide/Section3.4i.PNG
new file mode 100644
index 00000000..b261aa5d
Binary files /dev/null and b/docs/media/demo_guide/Section3.4i.PNG differ
diff --git a/docs/media/e0b725fd32e467f47cf879fa55a0a7b1.png b/docs/media/e0b725fd32e467f47cf879fa55a0a7b1.png
deleted file mode 100644
index ea2dc7b3..00000000
Binary files a/docs/media/e0b725fd32e467f47cf879fa55a0a7b1.png and /dev/null differ
diff --git a/docs/media/eac385448f2a333626219dbd74f113c8.png b/docs/media/eac385448f2a333626219dbd74f113c8.png
deleted file mode 100644
index fb43dc1a..00000000
Binary files a/docs/media/eac385448f2a333626219dbd74f113c8.png and /dev/null differ
diff --git a/docs/media/eae00fa728b99218930da021e4f58a6c.png b/docs/media/eae00fa728b99218930da021e4f58a6c.png
deleted file mode 100644
index bf23a666..00000000
Binary files a/docs/media/eae00fa728b99218930da021e4f58a6c.png and /dev/null differ
diff --git a/docs/media/user_documentation/1571073730733.png b/docs/media/user_documentation/1571073730733.png
new file mode 100644
index 00000000..355f4fe0
Binary files /dev/null and b/docs/media/user_documentation/1571073730733.png differ
diff --git a/docs/media/user_documentation/1571073733941.png b/docs/media/user_documentation/1571073733941.png
new file mode 100644
index 00000000..355f4fe0
Binary files /dev/null and b/docs/media/user_documentation/1571073733941.png differ
diff --git a/docs/media/user_documentation/1571079859530.png b/docs/media/user_documentation/1571079859530.png
new file mode 100644
index 00000000..94c65c36
Binary files /dev/null and b/docs/media/user_documentation/1571079859530.png differ
diff --git a/docs/media/user_documentation/1571081067899.png b/docs/media/user_documentation/1571081067899.png
new file mode 100644
index 00000000..a11d080b
Binary files /dev/null and b/docs/media/user_documentation/1571081067899.png differ
diff --git a/docs/media/user_documentation/1571081144550.png b/docs/media/user_documentation/1571081144550.png
new file mode 100644
index 00000000..59ff5a0f
Binary files /dev/null and b/docs/media/user_documentation/1571081144550.png differ
diff --git a/docs/media/user_documentation/1571087647200.png b/docs/media/user_documentation/1571087647200.png
new file mode 100644
index 00000000..12928161
Binary files /dev/null and b/docs/media/user_documentation/1571087647200.png differ
diff --git a/docs/media/user_documentation/1571088169759.png b/docs/media/user_documentation/1571088169759.png
new file mode 100644
index 00000000..24662e9a
Binary files /dev/null and b/docs/media/user_documentation/1571088169759.png differ
diff --git a/docs/media/user_documentation/1571088232926.png b/docs/media/user_documentation/1571088232926.png
new file mode 100644
index 00000000..808d73a7
Binary files /dev/null and b/docs/media/user_documentation/1571088232926.png differ
diff --git a/docs/release_notes.md b/docs/release_notes.md
index f0756404..a6f73160 100644
--- a/docs/release_notes.md
+++ b/docs/release_notes.md
@@ -1,12 +1,58 @@
-Relativity Trace Release Notes
-================================
-- [11.2.11.1 (13 September 2019)](#112111-13-september-2019)
+# Relativity Trace Release Notes
+
+- [12.0.5.2 (24 October 2019)](#12052-24-october-2019)
+- [11.2.11.1 (13 September 2019) - DEPRECATED](#112111-13-september-2019---deprecated)
- [11.2.10.1 (12 August 2019) - DEPRECATED](#112101-12-august-2019---deprecated)
- [11.2.9.2 (15 July 2019) - DEPRECATED](#11292-15-july-2019---deprecated)
- [11.2.6.1 (17 June 2019) - DEPRECATED](#11261-17-june-2019---deprecated)
- [11.2.4.4 (10 June 2019) - DEPRECATED](#11244-10-june-2019---deprecated)
-# 11.2.11.1 (13 September 2019)
+# 12.0.5.2 (24 October 2019)
+
+**Relativity Compatibility**
+
+- \> 9.6.202.10
+
+**Features**
+
+- New Data Enrichment Task performs data extraction independently of data retrieval, allows for better throughput and scalability of data ingestion pipeline
+- Trace Agents (Manager and Worker) now fully support resource pools which allows to designate multiple agent servers to scale out Trace workflow as needed
+
+**Enhancements**
+
+- Rule Type field can now be specified per rule. This allows to tag different MO field depending on rule configuration. Workflow rule configuration are used to drive workflow within the Relativity workspace
+ Alert rule configuration is used for standard e-com/a-com identification
+- Eliminated redundant query/read audits generated by Trace operations which reduces pressure on SQL server and audit infrastructure
+- All actions associated with any rule now operate on net new data that is reflected on dedicated saved search
+ - RelativityScript (advanced) action now operates on the same saved search, yielding incremental data only
+- Conceptual and Classification index automation now takes into account any in-progress activities that lock the index resulting in better robustness
+- dtSearch indexes will now perform Full Build automatically (no user interaction) on newly created indexes that are subject to automation
+- Data Transformation failures on set of docs (in data batch) now reports detailed per document failures and allows for partial import of successful transformations
+- Application installation time is now significantly faster, per workspace upgrade logic now is done by the Trace Manager agent
+- BIST testing coverage is now expanded to include:
+ - Explicit tests of successful Data Retention action execution
+ - Monitored Individual auto-discovery propagation
+ - Ingestion flow via Integration Points (as production)
+- Monitored Individuals can now take advantage of Secondary Identifier to accommodate variable identifiers across multiple data sources (see documentation)
+
+**Defect Fixes**
+
+- Long text truncation of ingestion state for data sources with large number of Monitored Individuals when updating via UI
+- Continuous BIST execution no longer generates orphaned resources and now correctly notifies of errors via Reporting task
+- BIST import dependencies were updated to be compatible with RelOne Goatsbeard (10.3+) release
+- Monitored Individual Auto-discovery no longer breaks the whole data batch on failures and reports errors on all extracted documents. Additionally, discovered monitored individuals are now properly associated with all documents that were extracted (including sub-attachments)
+
+**Deprecated**
+
+- Tag Action is now explicitly done as part of every rule evaluation and is not available in UI
+
+**Upgrade Considerations**
+
+- This is a **major** version change of Relativity Trace.
+
+# 11.2.11.1 (13 September 2019) - DEPRECATED
+
+> **DEPRECATION NOTE:** This version of Trace is **no longer supported**
**Relativity Compatibility**
diff --git a/docs/trace_connectors_user_guide.md b/docs/trace_connectors_user_guide.md
index 0a9d14bb..8002ef6d 100644
--- a/docs/trace_connectors_user_guide.md
+++ b/docs/trace_connectors_user_guide.md
@@ -47,7 +47,7 @@ For every Trace Data Source, except for [Microsoft Exchange Data Source](https:/
### Installation Steps
-Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to trace@relativity.com if you don't have access to this guide.
+Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to support@relativity.com if you don't have access to this guide.
### Data Flow Overview
@@ -122,7 +122,7 @@ Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to trace@relativity.com if
1. MUST configure Email Alert Settings
2. Send Test Email
3. 
- 8. For data source-specific instructions, Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to trace@relativity.com if you don't have access to this guide.
+ 8. For data source-specific instructions, Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to support@relativity.com if you don't have access to this guide.
9. Configure `Importer Schedule` to run at a desirable frequency (daily is the most common frequency)
diff --git a/docs/user_documentation.md b/docs/user_documentation.md
index e260c5d0..21410c9b 100644
--- a/docs/user_documentation.md
+++ b/docs/user_documentation.md
@@ -1,75 +1,74 @@
-Relativity Trace User Documentation
-================================
+# Relativity Trace User Documentation
- [Release Notes](#release-notes)
- [Introduction to Relativity Trace](#introduction-to-relativity-trace)
- [Prerequisites](#prerequisites)
- + [Agents](#agents)
- + [Applications](#applications)
+ - [Agents](#agents)
+ - [Applications](#applications)
+ - [BIST ( Built-in self-test ) Workspace](#bist--built-in-self-test--workspace)
- [Setting up Relativity Trace](#setting-up-relativity-trace)
- [Trace Document Flow Overview](#trace-document-flow-overview)
- * [Error Resolution Workflow and Retry](#error-resolution-workflow-and-retry)
+ - [Error Resolution Workflow and Retry](#error-resolution-workflow-and-retry)
- [Trace Rules Engine Overview](#trace-rules-engine-overview)
- * [1 - Creating a Rule](#1---creating-a-rule)
- * [2 - Customizing and Running a Rule](#2---customizing-and-running-a-rule)
- * [3 - Validating Results](#3---validating-results)
- * [Terms](#terms)
- + [Creating Terms](#creating-terms)
- + [Highlighting](#highlighting)
- * [Actions](#actions)
- + [Move To Folder Action Type](#move-to-folder-action-type)
- + [Data Archive Action Type](#data-archive-action-type)
- + [Advanced Action Type](#advanced-action-type)
- + [Alert Action Types](#alert-action-types)
- + [Replacement Tokens](#-replacement-tokens)
- - [Email Action Type](#email-action-type)
- - [Slack Action Type](#slack-action-type)
- - [Webhook Action Type (Preview)](#webhook-action-type--preview-)
- + [Custom RelativityScripts](#custom-relativityscripts)
+ - [1 - Creating a Rule](#1---creating-a-rule)
+ - [2 - Customizing and Running a Rule](#2---customizing-and-running-a-rule)
+ - [3 - Validating Results](#3---validating-results)
+ - [Terms](#terms)
+ - [Creating Terms](#creating-terms)
+ - [Highlighting](#highlighting)
+ - [Actions](#actions)
+ - [Move To Folder Action Type](#move-to-folder-action-type)
+ - [Data Archive Action Type](#data-archive-action-type)
+ - [Advanced Action Type](#advanced-action-type)
+ - [Alert Action Types](#alert-action-types)
+ - [Replacement Tokens](#replacement-tokens)
+ - [Email Action Type](#email-action-type)
+ - [Sample Email generated by Trace](#sample-email-generated-by-trace)
+ - [Slack Action Type](#slack-action-type)
+ - [Sample Slack message generated by Trace](#sample-slack-message-generated-by-trace)
+ - [Webhook Action Type (Preview)](#webhook-action-type-preview)
+ - [Custom Relativity Scripts](#custom-relativity-scripts)
- [Trace Proactive Ingestion Framework](#trace-proactive-ingestion-framework)
- * [Data Sources](#data-sources)
- + [Microsoft Exchange Data Source](#microsoft-exchange-data-source)
- + [Relativity Native Data Extraction Data Source](#relativity-native-data-extraction-data-source)
- * [Monitored Individuals](#monitored-individuals)
- * [Data Transformations](#data-transformations)
- + [Replace Data Transformation](#replace-data-transformation)
- + [Deduplication Data Transformation](#deduplication-data-transformation)
- * [Data Batches](#data-batches)
- * [Error Resolution Workflow and Retry](#error-resolution-workflow-and-retry-1)
- * [Automatic Discovery of Monitored Individuals](#automatic-discovery-of-monitored-individuals)
+ - [Data Sources](#data-sources)
+ - [Microsoft Exchange Data Source](#microsoft-exchange-data-source)
+ - [Relativity Native Data Extraction Data Source](#relativity-native-data-extraction-data-source)
+ - [Monitored Individuals](#monitored-individuals)
+ - [Data Transformations](#data-transformations)
+ - [Replace Data Transformation](#replace-data-transformation)
+ - [Deduplication Data Transformation](#deduplication-data-transformation)
+ - [Data Batches](#data-batches)
+ - [Error Resolution Workflow and Retry](#error-resolution-workflow-and-retry-1)
+ - [Automatic Discovery of Monitored Individuals](#automatic-discovery-of-monitored-individuals)
- [Setup](#setup)
- * [Tasks](#tasks)
- * [Alerts and Notifications](#alerts-and-notifications)
- * [Errors and Logging](#errors-and-logging)
+ - [Tasks](#tasks)
+ - [Alerts and Notifications](#alerts-and-notifications)
+ - [Errors and Logging](#errors-and-logging)
- [Built-In Self-Test (BIST)](#built-in-self-test-bist)
- [Reporting](#reporting)
- * [Trace Terms Report](#trace-terms-report)
+ - [Trace Terms Report](#trace-terms-report)
- [Considerations](#considerations)
- * [Usability Considerations](#usability-considerations)
- * [General Infrastructure and Environment Considerations](#general-infrastructure-and-environment-considerations)
- * [Large Workspaces Infrastructure and Environment Considerations](#large-workspaces-infrastructure-and-environment-considerations)
+ - [Usability Considerations](#usability-considerations)
+ - [General Infrastructure and Environment Considerations](#general-infrastructure-and-environment-considerations)
+ - [Large Workspaces Infrastructure and Environment Considerations](#large-workspaces-infrastructure-and-environment-considerations)
- [Glossary](#glossary)
- [Appendix A: Trace Object Architecture](#appendix-a-trace-object-architecture)
- [Appendix B: Trace Document Extraction Fields](#appendix-b-trace-document-extraction-fields)
- [Appendix C: Create Email Fields Map Integration Point Profile](#appendix-c-create-email-fields-map-integration-point-profile)
- * [Setup Integration Point Profile](#setup-integration-point-profile)
+ - [Setup Integration Point Profile](#setup-integration-point-profile)
Release Notes
================================
-| **Trace Version** | **Release Date** | **Relativity Compatibility** |
-| ----------------- | ---------------- | ---------------------------- |
-| 11.2.11.1 | 13 September 2019 | \> 9.6.202.10 |
+| Trace Version | Release Date | Relativity Compatibility |
+| ------------- | --------------- | ------------------------ |
+| 12.0.5.2 | 24 October 2019 | > 9.6.202.10 |
-More informaion is avaiable in [Relativity Trace Release Notes](https://relativitydev.github.io/relativity-trace-documentation/release_notes)
+More information is available in [Relativity Trace Release Notes](https://relativitydev.github.io/relativity-trace-documentation/release_notes)
Introduction to Relativity Trace
================================
-Relativity is primarily used for e-discovery, investigations and regulatory
-inquiries: typically reactive and transactional workflows. Relativity Trace is
-an application built on the Relativity platform for proactive compliance and
-surveillance workflows.
+Relativity is primarily used for e-discovery, investigations and regulatory inquiries: typically reactive and transactional workflows. Relativity Trace is an application built on the Relativity platform for proactive compliance and surveillance workflows.
Relativity Trace is an [ADS Deployable Application](https://platform.relativity.com/9.5/Content/Building_Relativity_applications/Building_Relativity_applications.htm) containing [RDOs](https://platform.relativity.com/9.5/Content/Managing_Relativity_dynamic_objects/RDO_9.5/Relativity_objects.htm), Custom [Agents](https://help.relativity.com/9.5/Content/System_Guides/Agents_Guide/Agents.htm),
[Event Handlers](https://help.relativity.com/9.5/Content/Relativity/Event_Handler_Express/Relativity_Event_Handler_Express.htm) and other relevant infrastructure.
@@ -77,8 +76,7 @@ Relativity Trace is an [ADS Deployable Application](https://platform.relativity.
Prerequisites
=============
-Ensure the default Relativity infrastructure has been setup and operational. In
-addition, Relativity Trace utilizes the following components for its processes:
+Ensure the default Relativity infrastructure has been set up and is fully operational. In addition, Relativity Trace utilizes the following components for its processes:
### Agents
@@ -92,13 +90,11 @@ addition, Relativity Trace utilizes the following components for its processes:
- Auto Batch Manager
-- Integration Points Agent (need to install [Relativity Integration Points](https://platform.relativity.com/9.6/Content/Relativity_Integration_Points/Get_started_with_integration_points.htm?)
- first)
-
-- Integration Points Manager (need to install [Relativity Integration Points](https://platform.relativity.com/9.6/Content/Relativity_Integration_Points/Get_started_with_integration_points.htm?)
- first)
+- Integration Points Agent (need to install [Relativity Integration Points](https://platform.relativity.com/9.6/Content/Relativity_Integration_Points/Get_started_with_integration_points.htm?) first)
+
+- Integration Points Manager (need to install [Relativity Integration Points](https://platform.relativity.com/9.6/Content/Relativity_Integration_Points/Get_started_with_integration_points.htm?) first)
-If you plan to use Analytics functionality, please also make sure the following
+If you plan to use `Analytics` functionality, please also make sure the following
agents are set up:
- Content Analyst Index Manager
@@ -113,11 +109,9 @@ agents are set up:
- Active Learning Worker
-- Structured Analytics Manager (need to install [Relativity Analytics](https://help.relativity.com/9.6/Content/Relativity/Analytics/Structured_analytics_set_tab.htm)
- first)
-
-- Structured Analytics Worker (need to install [Relativity Analytics](https://help.relativity.com/9.6/Content/Relativity/Analytics/Structured_analytics_set_tab.htm)
- first)
+- Structured Analytics Manager (need to install [Relativity Analytics](https://help.relativity.com/9.6/Content/Relativity/Analytics/Structured_analytics_set_tab.htm) first)
+
+- Structured Analytics Worker (need to install [Relativity Analytics](https://help.relativity.com/9.6/Content/Relativity/Analytics/Structured_analytics_set_tab.htm) first)
### Applications
@@ -125,84 +119,91 @@ agents are set up:
1. Used by Trace Data Sources
- 2. See this
- [page](https://help.relativity.com/9.6/Content/Relativity_Integration_Points/RIP_9.6/Installing_Integration_Points.htm)
- for details on how to install Integration Points
+ 2. See this [page](https://help.relativity.com/9.6/Content/Relativity_Integration_Points/RIP_9.6/Installing_Integration_Points.htm) for details on how to install Integration Points
+
+- [Relativity Analytics](https://help.relativity.com/9.6/Content/Relativity/Analytics/Structured_analytics_set_tab.htm)
+ 1. Used by Trace after ingestion to perform Structured Analytics workflows
+ (language identification, repeated content identification, etc)
+
+### BIST ( Built-in self-test ) Workspace
-- [Relativity
- Analytics](https://help.relativity.com/9.6/Content/Relativity/Analytics/Structured_analytics_set_tab.htm)
+1. Create new Relativity Workspace dedicated to BIST
- 1. Used by Trace after ingestion to perform Structured Analytics workflows
- (language identification, repeated content identification, etc)
+2. Follow steps in [Relativity Trace Automated Tests (BIST)](https://relativitydev.github.io/relativity-trace-documentation/bist_smoke_tests)
Setting up Relativity Trace
===========================
-1. [Install](https://help.relativity.com/9.6/Content/Relativity/Applications/Installing_applications.htm)
- the `Trace_.rap` from Application Library tab to all workspaces
+1. [Install](https://help.relativity.com/9.6/Content/Relativity/Applications/Installing_applications.htm) the `Trace_.rap` from Application Library (admin) tab to all workspaces
where needed
+
+2. Wait until application Status switches to `Installed` in the target workspaces
+
-2. Validate application install in the workspace with Trace application
- installed
-
- 
-
-3. Create Trace agent
-
- 1. Agent Type = `Trace Agent`
-
- 2. Number of Agents = `1`
-
- > **NOTE:** creating multiple Trace Agents in a Relativity instance is not supported and will cause failures and/or unpredictable behavior
-
- 3. Agent Server = Select the agent server you would like the agent deployed
- on (see “Infrastructure and Environment Considerations” section for
- optimal performance)
+
+
+3. Create Trace agents
- 4. Run Interval = `60`
+ > **NOTE:** Trace agents are Resource Pool aware. You are able to dedicate only one `Trace Manager Agent` per resource pool (associated with single workspace) and unlimited number of `Trace Worker Agents`
- 5. Logging level of event details = `Log all messages`
+ 1. Trace Manager Agent
+ 1. Agent Type = `Trace Manager Agent`
+ 2. Number of Agents = `1`
+ 3. Agent Server = Select the agent server you would like the agent deployed
+ on (see “Infrastructure and Environment Considerations” section for
+ optimal performance)
+ 4. Run Interval = `60`
+ 5. Logging level of event details = `Log all messages`
+ 2. Trace Worker Agent
+ 1. Agent Type = `Trace Worker Agent`
+ 2. Number of Agents = `Unlimited`
+ 3. Agent Server = Select the agent server you would like the agent deployed
+ on (see “Infrastructure and Environment Considerations” section for
+ optimal performance)
+ 4. Run Interval = `60`
+ 5. Logging level of event details = `Log all messages`
-4. Please review the
- [Considerations](#infrastructure-and-environment-considerations) for system
- impact information. By default system processes (Tasks) are scheduled to run
- every 5 minutes (configurable per workspace).
+4. Please review the
+ [Considerations](#infrastructure-and-environment-considerations) for system
+ impact information. By default system processes (Tasks) are scheduled to run
+ every 5 minutes (configurable per workspace).
- 1. Please reach out to `trace@relativity.com` for additional information
+ 1. Please reach out to `support@relativity.com` for additional information
-5. Configure Trace License
+5. On the `Agents` tab, view the Message of `Trace Manager Agent` until there are no longer any workspaces listed as `Updating` (this is necessary because the manager agent makes additional modifications to target workspaces after application install that are needed in the next steps) 
+> **NOTE:** On upgrades, the workspaces with existing data could take considerable time but should not take longer than 20-30 minutes to finish upgrading. Please reach out support@relativity.com if the upgrade takes longer.
- 1. At first install time, a default trial license is installed that is
- re-used for all workspaces in the Relativity instance (it’s valid 30
- days from installation date)
+6. Configure Trace License
- 2. You can request a new license via “Manage Trace License” link on Setup
- tab
+ 1. At first install time, a default trial license is installed that is
+ re-used for all workspaces in the Relativity instance (it’s valid 30
+ days from installation date)
- 
+ 2. You can request a new license via `Manage Trace License` link on Setup
+ tab
- 3. Click `Request Trace License` and send an email with the contents of the
- request to `trace@relativity.com`
+ 
- 
+ 3. Click `Request Trace License` and send an email with the contents of the
+ request to `support@relativity.com`
- > **WARNING:** Once license expires, all Trace processes stop working in all configured workspaces.
+ 
-6. In the workspace, navigate to the `Trace`->`Setup` tab and set the `Run Option`
- to `Continuous`
+ > **WARNING:** Once license expires, all Trace processes stop working in **all** configured workspaces.
- > **WARNING:** Changing the “Run Option” to “Continuous” will automatically build
- a dtSearch index for this workspace for all documents present. Only change this
- setting to "Continuous" when appropriate agent infrastructure is configured and
- disk space available to build a corresponding dtSearch Index. Please reach out
- to `trace@relativity.com` for support on installing Trace into workspaces with
- existing data.
+7. In the workspace, navigate to the `Trace`->`Setup` tab and set the `Run Option` to `Continuous`
+
+> **WARNING:** Changing the “Run Option” to “Continuous” will automatically build
+ a dtSearch index for this workspace for all documents present. Only change this
+ setting to "Continuous" when appropriate agent infrastructure is configured and
+ disk space available to build a corresponding dtSearch Index. Please reach out
+ to `support@relativity.com` for support on installing Trace into workspaces with
+ existing data.
Trace Document Flow Overview
============================
-Trace has a three-step process that all new documents go through. This status is
-tracked in the “Trace Document Status” field.
+Trace has a three-step process that all new documents go through. This status is tracked in the `Trace Document Status` field.
| | **Step 1** | **Step 2** | **Step 3** |
|-------------|---------------------------------------------------------------------------------|-----------------------------------------------|-----------------------------------------------------------------------|
@@ -214,98 +215,68 @@ status of documents is reflected on a few key fields on the Document object
1. **Trace Checkout** – Fixed-length Text field responsible for checking out
each document for the Trace Data Flow
-
2. **Trace Document Status** – Single choice field responsible for reflecting
overall progress of the document through the Trace Data Flow
-
- 1. Standard choices are “1 – New”, “2 – Indexed”, and “3 – Term Searched”
- reflecting the statuses above
-
- 2. “Indexing Errored” – reflects documents that have not successfully gone
- through Indexing. Potential causes:
-
- 1. Broken infrastructure (Agents, Service Host)
-
- 2. Documents could not be updated with new status due to SQL outage
-
- 3. Actions:
-
- 1. Check Setup tab for statuses of agents
-
- 2. Check Trace logs (via Manage Logs console button)
-
- 3. Perform Trace Document Retry mass-operation on affected
- documents
-
- 3. “Searching Errored” – reflects documents that have not successfully gone
- through Term Searching.
- Potential causes:
-
- 1. Broken infrastructure
-
- 1. Disabled agents
-
- 2. Stopped services (agent manager, service host)
-
- 2. Permanently broken/invalid Term syntax are present
-
+1. Standard choices are `1 – New`, `2 – Indexed`, and `3 – Term Searched` reflecting the statuses above
+
+ 2. `Indexing Errored` status reflects documents that have not successfully gone
+ through Indexing. Potential causes:
+1. **Broken infrastructure** (Agents, Service Host issues)
+
+2. Documents could not be updated with **new status** due to SQL outage
+
+3. **Actions**
+ 1. Check Setup tab for statuses of agents
+
+ 2. Check Trace logs (via `Manage Logs` console button)
+
+ 3. Perform `Trace Document Retry` mass-operation on affected documents
+
+ 3. `Searching Errored` status reflects documents that have not successfully gone
+ through Term Searching stage. Potential causes:
+ 1. Broken infrastructure (Agents, Service Host issues)
+ 2. Permanently broken/invalid Term (dtSearch) syntax are present
3. Actions:
-
1. Check Setup tab for statuses of agents
-
+
2. Check Terms tab for detailed errors on each failed term
-
- 3. Check Trace logs (via Manage Logs console button)
-
- 4. Perform Trace Document Retry mass-operation on affected
- documents
-
+
+ 3. Check Trace logs (via `Manage Logs` console button)
+
+ 4. Perform `Trace Document Retry` mass-operation on affected documents
3. **Trace Document Terms** – MultiObject field tracking which Terms have
matched a document
-
4. **Trace Document Rule Terms** – MultiObject field tracking Rule specific
Terms (terms that are associated with any rule) that have matched a document
-
5. **Trace Has Errors** – Boolean (yes/no) field indicating if the document has
any errors related to ingestion, extraction
-
6. **Trace Error Details** – Long Text field capturing the error details if a
document has encountered any errors
-
7. **Trace Data Transformations** – MultiObject field tracking what data
transformations have been applied to the document as part of ingestion
- (note, currently only data transformations of type Replace are attached to
- documents)
-
-8. **Trace Monitored Individuals –** MultiObject field tracking the people that
- are associated with the document (Monitored Individuals)
+8. **Trace Monitored Individuals –** MultiObject field tracking which Monitored Individuals are associated with each document
+9. **Trace Rules –** MultiObject field tracking which rule matched which document
You can get a quick understanding of the status of your system and documents by
applying appropriate aggregations and dashboards on these fields:
-> **NOTE:** Rule evaluation (including tagging documents to the Rule) is executed
-by the Rule Evaluation task and is outside of the core Trace Document Flow.
+> **NOTE:** Rule evaluation (including tagging documents to the Rule) is executed by the Rule Evaluation task and is outside of the core Trace Document Flow.
Error Resolution Workflow and Retry
-----------------------------------
If you wish to re-submit existing documents through the Trace Data Flow, you can
-accomplish this via Document mass operation “Trace Document Retry”. The mass
-operation resets the following fields: “Trace Checkout”, “Document Term”,
-“Rules” and “Trace Document Status”. Simply check the documents that you wish to
-retry from the Document List and click the item in the dropdown and click “Ok”
-on the pop-up. If your browser settings prevent pop-ups please enable them for
-Relativity URLs.
-
-**WARNING:** The retry process can be very resource-intensive. Trace is
-optimized for ongoing and forward-looking use cases where documents are only
-searched once upon ingestion. Triggering a retry will treat affected documents
-as if they were brand new to Trace, clearing all previous Rule and Term
-associations. If enough documents are retried at once, the system could struggle
-to handle the sudden influx of documents. Please exercise caution when using
-this feature.
+accomplish this via Document mass operation `Trace Document Retry`. The mass
+operation resets the following fields: `Trace Checkout`, `Trace Terms`, `Trace Rule Terms`,
+`Trace Rules`, `Trace Workflow Rules` and `Trace Document Status`. Simply check the
+documents that you wish to retry from the Document List and click the item in the
+dropdown and click `Ok` on the pop-up. If your browser settings prevent pop-ups please
+enable them for Relativity URLs.
+
+> **WARNING:** The retry process can be very resource-intensive. Trace is optimized for ongoing and forward-looking use cases where documents are only searched once upon ingestion. Triggering a retry will treat affected documents as if they were brand new to Trace, clearing all previous Rule and Term
+> associations. If enough documents are retried at once, the system could struggle to handle the sudden influx of documents. Please exercise caution when using this feature.
@@ -322,13 +293,18 @@ documents as they are ingested into the workspace.
1 - Creating a Rule
-------------------
-Create a new [rule](#_Glossary) by clicking `New Rule` on the Trace:Rules tab
+Create a new [rule](#_Glossary) by clicking `New Rule` on the `Trace`:`Rules` tab
-
+
The Rule Creation form contains the following fields:
-- **Name:** the name of the rule
+- **Rule Name:** the name of the rule
+
+- **Rule Type:** classification of the purpose of the rule
+
+ - **Alert:** a rule that is designed to alert on responsive documents. Documents matching rules of this type are tagged in the Trace Rules field.
+ - **Workflow:** a rule that is designed to operate on every document in the workspace and take some action (run a script, move to the correct folder, etc) but not to alert on responsive documents. Documents matching rules of this type are tagged in the Trace Workflow Rules field.
- **Searchable Set:** the document set searched when the Rule runs. Make sure
this set contains all documents that should be considered by Trace Rules.
@@ -336,22 +312,21 @@ The Rule Creation form contains the following fields:
- **Associated Actions:** actions are what happen when a rule matches a
document. Currently supported Action Types are:
- - **Tagging**: tags documents with the associated rule (happens
- automatically as part of Rule Evaluation)
-
- **Data Archiving**: triggers deletion of specified documents after the
configured retention policy
- **Advanced:** execute customer provided Relativity Script
-
- - **Email:** generates an email with metadata about alerted documents
-
- - **Slack:** generates a Slack message with metadata about alerted
+
+- **Email:** generates an email with metadata about alerted documents
+
+- **Slack:** generates a Slack message with metadata about alerted
documents
- **Webhook:** makes a generic API call hosted within Relativity
-
- - > **NOTE:** Batching action has been deprecated. You can still create a Batch Set manually from any saved search and set it to `auto-run`.
+
+- > **NOTE:** The Tagging action type has been deprecated. Documents are always tagged automatically with the associated Rule (happens as the final action as part of Rule Evaluation, if a document is tagged then all of the actions on the rule were executed)
+
+- > **NOTE:** Batching action has been deprecated. You can still create a Batch Set manually from any saved search and set it to `auto-run`.
2 - Customizing and Running a Rule
-----------------------------------
@@ -359,12 +334,12 @@ The Rule Creation form contains the following fields:
After you create a Rule, use the Rule definition page to Enable and Customize
the rule.
-
+
The Trace Rule Management Console on the right-hand side has the following
buttons:
-- **Enable Rule:** enabling the rule allows agents to run rule on an ongoing
+- **Enable Rule:** enabling the rule allows Trace Manager agent to run rule on an ongoing
basis
> **NOTE:** A Rule will only execute when it is enabled.
@@ -380,22 +355,19 @@ associated with the Rule during evaluation.
After Rule executes, documents matching the Rule will be associated to the Rule
itself.
-
+
Terms
-----
-In addition to metadata conditions of a Saved Search associated with Rule, you
-can apply extensive searching criteria to isolate only the most relevant
-documents.
+In addition to metadata conditions of a Saved Search associated with Rule, you can apply extensive searching criteria to isolate only the most relevant documents.
### Creating Terms
You can create Terms in multiple ways:
-1. Via [Remote Desktop Client ( RDC )](https://help.relativity.com/9.6/Content/Relativity/Relativity_Desktop_Client/Relativity_Desktop_Client.htm)
- load file. This method is ideal if you are adding a lot of terms at once.
-
+1. Via [Remote Desktop Client ( RDC )](https://help.relativity.com/9.6/Content/Relativity/Relativity_Desktop_Client/Relativity_Desktop_Client.htm) load file. This method is ideal if you are adding a lot of terms at once.
+
2. From Terms tab by clicking “New Term” and adding each Term individually.
3. By clicking the New button on the Terms section of the Rule Layout.
@@ -405,10 +377,9 @@ You can create Terms in multiple ways:
Term definition contains 3 fields:
- **Term:** searching string (unique identifier)
-
- **Term Category:** optional group name to organize terms
-
-- **Relativity highlight color:** specifies highlighting configuration in the
+- **Score:** optional field to populate term weight to indicate importance
+- **Relativity highlight color:** optional highlighting configuration in the
Relativity viewer (see [highlighting](#highlighting) section)
In addition, you can see and modify Term Categories and Rules associated with
@@ -416,62 +387,48 @@ Term and its status with regards to execution
-> **NOTE:** The Term “Name” (actual text being searched) **cannot** be modified
-after it is created. You must remove and add a new term object to change the
-search string. You **can** modify the highlight color and term category of an
-existing term.
+> **NOTE:** The Term “Name” (actual text being searched) **cannot** be modified after it is created. You must remove and add a new term object to change the search string. You **can** modify the highlight color and term category of an existing term.
-> **NOTE: The** Term “Name” (actual text being searched) is limited to 450
-characters. Please reach out `trace@relativity.com` if your use-case requires
-higher limits for your terms.
+> **NOTE: The** Term “Name” (actual text being searched) is limited to 450 characters. Please reach out `support@relativity.com` if your use-case requires higher limits for your terms.
### Highlighting
-By default, Trace creates a “Trace Persistent Highlight Set” that is populated
-with **all** terms present in the workspace. In addition, the “Trace Rules
-Persistent Highlight Set” captures only terms currently associated with any
+By default, Trace creates a `Trace Persistent Highlight Set` that is populated
+with **all** terms present in the workspace. In addition, the `Trace Rules
+Persistent Highlight Set` captures only terms currently associated with any
Rule. You can have many terms you want to use for highlighting purposes only,
and not necessarily as part of matching any specific Rule. You can adjust the
order in which they are displayed in the viewer.
+
+
-In addition, you can override default highlight configuration (magenta
-background and black text) by specifying a semi-colon separated list of
-pre-configured color combinations. The details of the color codes can be
-accessed via Context Help button on the Term Definition page.
+
+
+In addition, you can override default highlight configuration (magenta background and black text) by specifying a semi-colon separated list of pre-configured color combinations. The details of the color codes can be accessed via Context Help button on the Term Definition page.
Actions
-------
-As part of installation, Action Types are created. Currently supported action
-types are: Tag, Data Archive, Advanced, Email, Slack, and Webhook and Move To
-Folder. For each Action Type there is a Default Action created. You can also
-customize and configure your own Actions.
+As part of installation, Action Types are created. Currently supported action types are: Data Archive, Advanced, Email, Slack, and Webhook and Move To Folder. For each Action Type there is a Default Action created. You can also customize the Default Actions and also create and configure your own Actions.
### Move To Folder Action Type
-Move To Folder action works on documents that match Rule criteria. Upon
-execution of the action, the documents will be moved to specified document
-folder, inheriting folder permissions. This action can be used to effectively
-secure the documents to specific set of users/groups. This can also be used to
-drive regional review workflow in conjunction with alerting actions.
+Move To Folder action works on documents that match Rule criteria. Upon execution of the action, the documents will be moved to the specified destination folder, inheriting folder permissions. This action can be used to effectively secure documents to specific set of users/groups by routing them to folders with
+different permission sets. This Action Type can also be used to drive regional review workflows in conjunction with alerting actions.
-You can configure the action by specifying the ArtifactID of the destination
-folder where the documents are to be moved. Additional Information section
-automatically populates the list of available folders and their corresponding
-ArtifactIDs for your convenience.
+You can configure the action by specifying the ArtifactID of the destination folder where the documents are to be moved. The Additional Information section automatically populates the list of available folders and their corresponding Artifact IDs for your convenience.
### Data Archive Action Type
-> **WARNING:** The Data Archive Action will permanently delete all documents that
-match the Rule conditions and are outside the Data Retention window.
+> **WARNING:** The Data Archive Action will permanently delete all documents that match the Rule conditions and are outside the Data Retention window.
The Data Archive Action Type follows the same Trace Rules Engine paradigm with
one added condition:
@@ -484,21 +441,18 @@ one added condition:
- Term Searching (optional)
- - *SPECIFIC TO ARCHIVE*: Workspace Data Retention policy
+ - *SPECIFIC TO ARCHIVE*: `Delete Documents Older Than Hours` setting
-Any data that is within the data retention policy (from the System Created On
-date/time) will not be deleted even if the document is included in the
+Any document that is newer than the specified number of hours (based on System
+Created On date/time) will not be deleted even if the document is included in the
Searchable Set / Search Term
-> **NOTE:** By default, the Data Archive Action will delete 1,000 documents per
-Run Interval. To adjust configurations so documents can be deleted more
-frequently, reach out to `trace@relativity.com`
-
**Action Configuration**
`Document Delete Batch Size` - controls number of documents to delete at once
-`Delete Documents Older Than Hours` - controls age of a document (based on creation of document in Relativity) to delete
+`Delete Documents Older Than Hours` - controls age of a document (based on System
+Created On date/time) to delete
### Advanced Action Type
@@ -506,9 +460,7 @@ frequently, reach out to `trace@relativity.com`
Scripts. Apply rigorous testing and impact assessment prior to deploying any
custom script in production or enabling it to run continuously via Trace
automation. For more information about Relativity Scripts in general, see the
-[Scripts](https://help.relativity.com/9.6/Content/Relativity/Scripts.htm) and
-[Scripts Properties](https://platform.relativity.com/9.6/Content/Scripts/Script_properties/Script_properties.htm)
-documentation pages.
+[Scripts](https://help.relativity.com/9.6/Content/Relativity/Scripts.htm) and [Scripts Properties](https://platform.relativity.com/9.6/Content/Scripts/Script_properties/Script_properties.htm) documentation pages.
The Advanced Action Type executes a Relativity Script automatically on a
recurring basis per the Rule Evaluation task configuration (schedule).
@@ -522,11 +474,11 @@ one yourself.
Search”
- For more information about Relativity Script feature in general, see the
- [Scripts](https://help.relativity.com/9.6/Content/Relativity/Scripts.htm) and [Scripts Properties](https://platform.relativity.com/9.6/Content/Scripts/Script_properties/Script_properties.htm) documentation pages.
+ [Scripts](https://help.relativity.com/9.6/Content/Relativity/Scripts.htm) and [Scripts Properties](https://platform.relativity.com/9.6/Content/Scripts/Script_properties/Script_properties.htm) documentation pages.
> **NOTE:** Currently all Relativity Scripts that can be associated with Trace
actions require a Saved Search as one of the script inputs (Trace automatically
-populates that field during Rule Evaluation with Rule specific saved search).
+populates that field during Rule Evaluation with an execution specific saved search).
@@ -561,16 +513,15 @@ For reference:
- “Trace Hour Of Day”, and “Trace Day Of Week” are SQL Column names of
corresponding Relativity Fields
-> **NOTE:** Trace will automatically create a Saved Search that includes a
-combination of your chosen Saved Search (from the Rule) AND your Term conditions
-and use that as the input for the script.
+> **NOTE:** Trace will automatically create a Saved Search that returns the net-new documents
+from your chosen Saved Search (from the Rule) AND your Term conditions and use that as
+the input for the script.
**Step 3**: To execute your script, attach this Action to a Rule and enable it
(as you would with any Rule).
-> **NOTE:** Advanced actions will run on a schedule, continuously. For scripts
-that have the potential to use significant resources, we recommend designing the
-script to be aware of what has already been acted on.
+> **NOTE:** Advanced actions will run on a schedule, continuously. Please consider the resource
+usage of your scripts!
### Alert Action Types
@@ -583,33 +534,23 @@ You can specify Trace Replacement Tokens in most configuration fields for the
Alert Action Types. These tokens will be replaced with information relevant to
the specific document, rule and/or alert:
-`<>` - Link to view Trace Rule in Relativity
-that generated the alert
+`<>` - Link to view Trace Rule in Relativity that generated the alert
-`<>` - Link to view Document matched by the
-alert
+`<>` - Link to view Document matched by the alert
-`<>` - Relativity document identifier
-(Control Number)
+`<>` - Relativity document identifier (Control Number)
-`<>` - ArtifactID of the document matched
-by the alert
+`<>` - ArtifactID of the document matched by the alert
-`<>` - Name of the Trace Rule that generated the
-alert
+`<>` - Name of the Trace Rule that generated the alert
-`<>` - Relativity Workspace ID (ArtifactID) of the
-workspace that generated the alert
+`<>` - Relativity Workspace ID (ArtifactID) of the workspace that generated the alert
> **NOTE:** The Trace Relativity Replacement tokens are not configurable, and are the only tokens available in the Trace application.
#### Email Action Type
-You can configure the Email action to send out an email about specific document
-matching rule conditions. Note that the Email Action uses the SMTPUserName,
-SMTPPassword, SMTPServer, SMTPPort and SMTPSSLisRequired settings from the
-kCura.Notification Section of Instance Settings to send your email messages, so
-be sure they are configured properly.
+You can configure the Email action to send out an email about specific document matching rule conditions. Note that the Email Action uses the SMTPUserName, SMTPPassword, SMTPServer, SMTPPort and SMTPSSLisRequired settings from the kCura.Notification Section of Instance Settings to send your email messages, so be sure they are configured properly.
@@ -693,9 +634,9 @@ combination with Base Api Url)
Trace agent. The access_token is retrieved from
ClaimsPrincipal.Current.Identities.
-### Custom RelativityScripts
+### Custom Relativity Scripts
-Several useful SQL RelativityScripts are shipped by default with Trace
+Several useful SQL Relativity Scripts are shipped by default with Trace
application.
| **Script Name** | **Description** | **Inputs and Outputs** |
@@ -707,8 +648,7 @@ Trace Proactive Ingestion Framework
The Proactive Ingestion Framework allows Administrators to automatically and
continually ingest data into Relativity from various Data Sources. The framework
-is built on top of [Relativity Integration
-Points](https://help.relativity.com/9.6/Content/Relativity_Integration_Points/RIP_9.6/Installing_Integration_Points.htm).
+is built on top of [Relativity Integration Points](https://help.relativity.com/9.6/Content/Relativity_Integration_Points/RIP_9.6/Installing_Integration_Points.htm).
The key benefits of the Proactive Ingestion Framework include:
@@ -717,7 +657,7 @@ The key benefits of the Proactive Ingestion Framework include:
- Data can be autonomously and continuously ingested
-- Data can be pushed from a 3PrdP party data processor in a generic format
+- Data can be pushed from a 3rd party data processor in a generic format
- Data is broken into batches improving stability and throughput
@@ -732,18 +672,14 @@ The key benefits of the Proactive Ingestion Framework include:
- Performance monitoring of entire data ingestion pipeline (bottleneck
identification, SLA metrics, proactive alerting)
-Reach out to for help integrating with the Proactive
+Reach out to for help integrating with the Proactive
Ingestion Framework
Data Sources
------------
-Data Source is a Relativity Dynamic Object (RDO) that ships with Trace
-application. It allows you to define where/how you are pulling data. The Data
-Source references the Integration Point Profile that holds configuration on how
-to import data for that Data Source (field mappings). Data Batches reference
-Data Source to dynamically lookup which Integration Profile to use during
-import.
+Data Source is a Relativity Dynamic Object (RDO) that ships with Trace application. It allows you to define where/how you are pulling data. The Data Source references the Integration Point Profile that holds configuration on how to import data for that Data Source (field mappings). Data Batches reference
+Data Source to dynamically lookup which Integration Profile to use during import.
@@ -794,12 +730,8 @@ The following are fields on the Data Source Object:
### Microsoft Exchange Data Source
-The Microsoft Exchange Data Source enables Relativity to automatically pull
-emails from a Microsoft Exchange instance (Office 365 or On Premises) into
-Relativity. The Microsoft Exchange Data Source is executed by the Data Retrieval
-task (seen on the Setup tab). Note, this Data Source only pulls emails at this
-time, if you need to retrieve other object types from Microsoft Exchange please
-reach out to `trace@relativity.com`
+The Microsoft Exchange Data Source enables Relativity to automatically pull emails from a Microsoft Exchange instance (Office 365 or On Premises) into Relativity. The Microsoft Exchange Data Source is executed by the Data Retrieval task (seen on the Setup tab). Note, this Data Source only pulls emails at this
+time, if you need to retrieve other object types from Microsoft Exchange please reach out to `support@relativity.com`
**Setup**
@@ -898,8 +830,7 @@ are automatically extracted into individual documents – e.g. zip with 10 word
(.docx) documents = 11 Relativity documents. In addition, images from email
content and each individual document are automatically expanded into separate
Relativity documents.
-> **NOTE:** the Microsoft Exchange data source only retrieves
-emails. It does not retrieve other exchange metadata at this time.
+> **NOTE:** the Microsoft Exchange data source only retrieves emails. It does not retrieve other exchange metadata at this time.
Please, refer to [Appendix B: Trace Document Extraction Fields](#appendix-b-trace-document-extraction-fields) for field descriptions.
@@ -971,92 +902,55 @@ by Relativity Trace. Generally a Relativity Trace license will specify a number
of Monitored Individuals available and the number of data sources they can be
used on.
-> **NOTE:** The only field on Monitored Individual currently used in application
-> logic is `Identifier` field. All other fields are simply for display purposes. Each
-> Monitored Individual must have a unique value in the Identifier field. Typically
-> the Identifier is the employee’s email address. Identifier is **case-sensitive**
-> (e.g. `Test@test.com` and `test@test.com` are treated as two different email
-> addresses / identifiers)
+> **NOTE:** The only field on Monitored Individual currently used in application logic is `Identifier` field. All other fields are simply for display purposes. Each Monitored Individual must have a unique value in the Identifier field. Typically the Identifier is the employee’s email address. Identifier is **case-sensitive**
+> (e.g. `Test@test.com` and `test@test.com` are treated as two different email addresses / identifiers)
Data Transformations
--------------------
-Trace Data Transformation is a Relativity Dynamic Object (RDO) that ships with
-Trace application. It allows administrators to specify a way data should be
-transformed while it is being imported by the data source. Currently there are
-two types of Trace Data Transformations, Replace and Deduplication. Trace Data
-Transformations are attached to a Data Source by clicking Link in the Data
-Transformations section of the Data Source Layout.
+Trace Data Transformation is a Relativity Dynamic Object (RDO) that ships with Trace application. It allows administrators to specify a way data should be transformed while it is being imported by the data source. Currently there are two types of Trace Data Transformations, Replace and Deduplication. Trace Data
+Transformations are attached to a Data Source by clicking Link in the Data Transformations section of the Data Source Layout.
### Replace Data Transformation
-Data Transformations of type *Replace* allow you to strip exact blocks of text
-out of the Extracted Text for any documents imported by the associated Data
-Source. This allows for removal of things like email signatures and other
-frequently occurring, benign text so that they do not match Terms. A Data Source
+Data Transformations of type *Replace* allow you to strip exact blocks of text out of the Extracted Text for any documents imported by the associated Data Source. This allows for removal of things like email signatures and other frequently occurring, benign text so that they do not match Terms. A Data Source
can be associated with multiple Data Transformations of type Replace.
-By Default, content will simply be replaced with nothing (empty string). This
-can be changed by entering Match Replacement Text in the Configuration section.
+By Default, content will simply be replaced with nothing (empty string). This can be changed by entering Match Replacement Text in the Configuration section.
### Deduplication Data Transformation
-Data Transformations of type *Deduplication* prevent a Data Source from
-importing a document if the same document already exists in the workspace. Only
-one Data Transformation of type Deduplication should be associated with each
-Data Source.
+Data Transformations of type *Deduplication* prevent a Data Source from importing a document if the same document already exists in the workspace. Only one Data Transformation of type Deduplication should be associated with each Data Source.
-Deduplication is driven by a SHA256 hashing algorithm that populates the Trace
-Document Hash field on each document. By Default, if the document is an email,
-the algorithm will hash together the sender, subject, recipients, sent date,
-email body and attachment list to create the hash value. If the document is not
-an email, then the hash will be done directly on the bytes of the file. It is
-possible to configure the exact hashing algorithm used for emails using the
-settings in the Configuration section:
+Deduplication is driven by a SHA256 hashing algorithm that populates the Trace Document Hash field on each document. By Default, if the document is an email, the algorithm will hash together the sender, subject, recipients, sent date, email body and attachment list to create the hash value. If the document is not an email, then the hash will be done directly on the bytes of the file. It is possible to configure the exact hashing algorithm used for emails using the settings in the Configuration section:
-When additional documents are ingested (either within the same Data Batch or
-different Data Batches), hashes will be compared to those on documents that
-already exist in the workspace. If there is a match, the duplicate document will
-not be ingested. Instead, the Trace Monitored Individuals field on the document
-will be updated to include the Monitored Individual that was the source of the
-duplicate in addition to the Monitored Individual that was the source of the
-original.
+When additional documents are ingested (either within the same Data Batch or different Data Batches), hashes will be compared to those on documents that already exist in the workspace. If there is a match, the duplicate document will not be ingested. Instead, the Trace Monitored Individuals field on the document
+will be updated to include the Monitored Individual that was the source of the duplicate in addition to the Monitored Individual that was the source of the original.
Data Batches
------------
-Data batch is a unit of ingestion work for Trace. It corresponds to a load file
-on disk that needs to be imported with specific settings and field mappings. It
-is a central tracking object for how the data was generated, normalized and
-aggregated into load file. It provides status of the overall import process and
-allows for deep audit history trail from “source” to Relativity.
+Data batch is a unit of ingestion work for Trace. It corresponds to a load file on disk that needs to be imported with specific settings and field mappings. It is a central tracking object for how the data was generated, normalized and aggregated into load file. It provides status of the overall import process and
+allows for deep audit history trail from native data source to Relativity.
-Once a batch enters Ingestion process (when status is set to: “Ready For
-Import”) Ingestion task will create an integration point from the configured
-Data Source profile (this information includes import settings and field
-mappings).
+Once a batch enters Ingestion process (when status is set to: `Ready For Import`) Ingestion task will create an integration point from the configured Data Source profile (this information includes import settings and field mappings).
Error Resolution Workflow and Retry
-----------------------------------
-By default the batch will be automatically retried internally (up to 3 times) if
-the data batch import was not completed. You can get the error details by
-looking at “Error Details” of a specific batch with “Has Errors” set to yes. You
-can manually mark the batch as “Abandoned” to indicate that it was manually
-resolved or “Retry” it.
+By default the batch will be automatically retried internally (up to 3 times) if the data batch import was not completed. You can get the error details by looking at `Error Details` of a specific batch with `Has Errors` set to yes. You can manually mark the batch as `Abandoned` to indicate that it was manually resolved or retry it.
-Data Batch objects have associated Mass Operations (and corresponding Data Batch
-console UI buttons) to help with state resolution
+Data Batch objects have associated Mass Operations (and corresponding Data Batch console UI buttons) to help with state resolution
-1. Retry – submit the batch to be retried by Trace
+1. `Trace Data Batch Retry` – submit the batch to be retried by Trace
-2. Abandon – indicates the batch has been manually resolved and should not be
+2. `Trace Data Batch Abandon` – update the batch to indeicate that it has been manually resolved and should not be
treated as errored.
@@ -1064,59 +958,58 @@ console UI buttons) to help with state resolution
Automatic Discovery of Monitored Individuals
--------------------------------------------
-For Data Source that are not shipped with Trace natively, application allows for
-automatic discovery of monitored individuals based on EmailFrom, EmailTo,
-EmailCC and EmailBCC fields.
+For Data Sources that are not shipped with Trace natively, the application allows for automatic discovery of monitored individuals based on `EmailFrom`, `EmailTo`, `EmailCC` and `EmailBCC` fields.
-Please, contact `trace@relativity.com` in order to enable this feature.
+Please, contact `support@relativity.com` in order to enable this feature.
Setup
=====
-The Setup tab aggregates the most important information about configuration,
-health and overall status of Trace for a workspace. It shows active tasks and
-their configuration, and a snapshot of instance infrastructure that’s relevant
-to Trace. In addition, you can manage Logging and License configuration and run
-a built-in self-test (BIST) to verify basic flow.
+The `Setup` tab aggregates the most important information about configuration, health and overall status of Trace for a workspace. It shows the currently installed version, active tasks and their configuration, and a snapshot of instance infrastructure that’s relevant to Trace. In addition, you can manage Logging and License configuration and run a built-in self-test (`BIST`) to verify basic flow.
-
+
+
+If an update to a new version of Trace is in progress, the Setup page will show a large red bar indicating an Update is in progress:
+
+
Tasks
------
-Tasks are ongoing background processes that are triggered by agents and run on
-the agent servers. They each have Run Intervals and configurations can be
-adjusted at a workspace level.
+Tasks are ongoing background processes that are triggered by agents and run on the agent servers. They each have Run Intervals and configurations can be adjusted at a workspace level.
-Each task is designed to be auto-recoverable and self-healing. For example, if there are temporary network connection issues that prevent `Data Retrieval` task from retrieving data, Trace will keep trying until network issues are resolved. No manual intevention is needed.
+Each task is designed to be auto-recoverable and self-healing. For example, if there are temporary network connection issues that prevent `Data Retrieval` task from retrieving data, Trace will keep trying until network issues are resolved. No manual intervention is needed.
-- **Data Retrieval:** Responsible for pulling data for Data Sources and
- creation of Data Batches
+- **Data Retrieval:** Responsible for pulling data for Data Sources
- **Ingestion:** Responsible for triggering import of the Data Batches into
Relativity (part of Proactive Ingestion Framework). Any Data Transformations
configured for the corresponding Data Source will also be performed.
-
+
- **Ingestion Status:** Responsible for updating statuses of the Data Batches
(part of Proactive Ingestion Framework)
-
-- **Indexing:** Responsible for indexing data needed for searching
+
+- **Indexing:** Responsible for indexing data needed for searching
- **Term Searching:** Responsible for executing searching of the Terms for
Rule evaluations
-
+
- **Rule Evaluation:** Responsible for evaluating configured Rules within the
workspace
-
+
- **Reporting**: Responsible for reporting on the state of the system via
email
+
+- **Data Enrichment:** Responsible for extracting nested files (attachments, contents of zip files), generating extracted text and preparing the loadfile that is ready for import process
+
+ > **NOTE:** Data Enrichment task queues up work via SQL based ServiceBus. Actual enrichment work is performed by `Trace Worker Agent`
Alerts and Notifications
------------------------
-By default Reporting task will send out the system health report to the
+By default the Reporting task will send out the system health report to the
configured instance email address every 24 hours. The defaults can be overridden
under Task Configuration on the Reporting task page. See below sample
configuration for example.
@@ -1159,27 +1052,17 @@ list.
-> **CAUTION:** Verbose levels (information/debug) of Logging framework can
-generate substantial load to default SQL configuration EDDSLogging database.
-Don’t forget to adjust the levels once low level information is no longer
-needed.
+> **CAUTION:** Verbose levels (information/debug) of Logging framework can generate substantial load to default SQL configuration EDDSLogging database. Don’t forget to adjust the levels once low level information is no longer needed.
Built-In Self-Test (BIST)
=========================
-Built-In Self-Test (BIST) is a separate Trace Task that can be enabled in
-certain workspaces. By default, BIST does a basic “happy path” test of the
-majority of Trace functionality to make sure Trace is functioning properly. For
-more details including instructions on how to enable BIST in a workspace, please
-refer to the separate [Relativity Trace Automated Tests (BIST)](https://relativitydev.github.io/relativity-trace-documentation/bist_smoke_tests), which describes
-all of the different tests users can run to ensure that Relativity and Trace
-have been set up and configured properly.
+Built-In Self-Test (BIST) is a separate Trace Task that can be enabled in certain workspaces. By default, BIST does a basic “happy path” test of the majority of Trace functionality to make sure Trace is functioning properly. For more details including instructions on how to enable BIST in a workspace, please refer to the separate [Relativity Trace Automated Tests (BIST)](https://relativitydev.github.io/relativity-trace-documentation/bist_smoke_tests), which describes all of the different tests users can run to ensure that Relativity and Trace have been set up and configured properly.
Reporting
=========
-`Trace`->`Reports` tab serves as a place for reporting capabilities. We will be
-adding more reports to this section in the future.
+`Trace`->`Reports` tab serves as a place for reporting capabilities. We will be adding more reports to this section in the future.
Trace Terms Report
------------------
@@ -1195,59 +1078,39 @@ Usability Considerations
------------------------
-- Once a document is associated with a Rule, it will never be disassociated
- unless there are document updates to extracted text or metadata. Manual
- Trace Document Retry (mass operation) procedure will also reset the
- associations automatically.
-
-- Every Data Source has capability to be “reset”. Once a data source is reset,
- Trace will pull all available data again, beginning with the Start Date
- defined on the data source (if the Start Date is relevant to the data source
- type). **Depending on import profile settings, this could duplicate data in
- the Workspace.**
-
-- Task processes (Indexing, Term Searching, Rule Evaluation, etc) run
- asynchronously from each other. It may take several task cycles (based on
- configured ”Run Intervals”) for the end-to-end workflow to complete fully.
-
-- Deleting a Trace Rule does not delete any of the corresponding Relativity
- infrastructure and objects that were created (i.e. dtSearch Index, Saved
- Searches, Batch Sets).
-
-- The global dtSearch index “Trace Search Index” (created by Trace application
- during installation) is supported for ad-hoc searching and will be
- incrementally built as part of Indexing task. No other dtSearch indexes will
- incrementally build automatically.
-
+- Once a document is associated with a Rule, it will never be disassociated unless there are document updates to extracted text or metadata. Manual `Trace Document Retry` (mass operation) procedure will also reset the associations automatically.
+
+- Every Data Source has capability to be `Reset` via console buttons. Once a data source is reset,
+ Trace will pull all available data again, beginning with the Start Date defined on the data source (if the Start Date is relevant to the data source type). **Depending on import profile settings, this could duplicate data in the Workspace.**
+
+- Task processes (Indexing, Term Searching, Rule Evaluation, etc) run asynchronously from each other. It may take several task cycles (based on configured `Run Interval`) for the end-to-end workflow to complete fully.
+
+- Deleting a Trace Rule does not delete any of the corresponding Relativity infrastructure and objects that were created (i.e. dtSearch Index, Saved Searches, Batch Sets).
+
+- The global dtSearch index `Trace Search Index` (created by Trace application during installation) is supported for ad-hoc searching and will be incrementally built as part of Indexing task. No other dtSearch indexes will incrementally build automatically.
+
- If Analytics Server is present, active and configured for the Relativity
instance:
- - If enabled, both “Trace Conceptual Analytics Index” and “Trace
- Classification Analytics Index” will incrementally build automatically.
- In order to enable this functionality, manually perform Full Build on
- each index.
+ - If enabled, both `Trace Conceptual Analytics Index` and `Trace Classification Analytics Index` will incrementally build automatically. In order to enable this functionality, manually perform Full Build on each index.
+
+ - If enabled, `Trace Structured Analytics Set` will run on incremental data performing specified job options (by default: Language Identification). In order to enable this functionality, manually perform Full Analysis.
+ + When [setting up Language Identification](https://help.relativity.com/RelativityOne/Content/Relativity/Analytics/Running_structured_data_analytics.htm#Creating_a_structured_analytics_set), for the document set, configure the saved search's condition to `Primary language` = `is not set`. This will ensure that only newly ingested documents are flagged for analysis.
- - If enabled, “Trace Structured Analytics Set” will run on incremental
- data performing specified job options (by default: Language
- Identification). In order to enable this functionality, manually perform
- Full Analysis.
-
- + When [setting up Language Identification](https://help.relativity.com/RelativityOne/Content/Relativity/Analytics/Running_structured_data_analytics.htm#Creating_a_structured_analytics_set), for the document set, configure the saved search's condition to `Primary languag` = `is not set`. This will ensure that only newly ingested documents are flagged for analysis.
-
- 
+ 
General Infrastructure and Environment Considerations
---------------------------------------------
| **Tasks:** | **Ingestion** | **Ingestion** | **Running Rules** | **Running Rules** |
|----------------------------------|----------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------|----------------------------------------------------------------------|
-| | **`Ingestion`, `Ingestion Status`, and `Data Retrieval`** | **`Indexing`** | **`Term Searching`** | **`Rule Evaluation`** |
+| | **`Ingestion`, `Ingestion Status`, `Data Enrichment` and `Data Retrieval`** | **`Indexing`** | **`Term Searching`** | **`Rule Evaluation`** |
| Summary | Powers the Proactive Ingestion Framework | Incrementally builds Trace Search Index and Temporary Search Indexes with batches of documents Optionally: Incrementally builds and runs Analytics Jobs (Conceptual, Classification and Structured) | Searches and tags Terms in workspace on an ongoing basis | Rule will run and tag all matching documents and perform actions |
-| Task Operations | *`Ingestion Task`*
-Looks for batches that need to be imported and kicks off import
-Creates RIP job per batch
*`Ingestion Status Task`*
-Updates Data Batch status
*`Data Retrieval Task`*
-Pulls data for enabled Data Sources executed by Trace Agent | *`Indexing Task`*
-Kicks off dtSearch incremental build
-Kicks off temporary (internal) dtSearch index builds for Term evaluation
-Kicks off Analytics Jobs (if configured)
**NOTE:** Run Interval controls frequency of temp indexes creation ONLY. Global dtSearch and Analytics Indexes are built every 60 minutes by default. | *`Term Searching Task`*
-Runs (searches and tags) Terms in the workspace
**NOTE:** Each Term Searching run interval will search up to 5 available document batches (default 10,000 documents per batch). These settings are configurable on Term Searching task. | *`Rule Evaluation Task`*
-Evaluates each rule in the workspaces and triggers configured actions |
+| Task Operations | *`Ingestion Task`*
-Looks for batches that need to be imported and kicks off import
-Creates RIP job per batch
*`Ingestion Status Task`*
-Updates Data Batch status
*`Data Retrieval Task`*
-Pulls data for enabled Data Sources executed by `Trace Manager Agent`
`Data Enrichment Task` extracts/expands/enriches source native document and prepares import load file (executed by `Trace Worker Agent`) | *`Indexing Task`*
-Kicks off dtSearch incremental build
-Kicks off temporary (internal) dtSearch index builds for Term evaluation
-Kicks off Analytics Jobs (if configured)
**NOTE:** Run Interval controls frequency of temp indexes creation ONLY. Global dtSearch and Analytics Indexes are built every 60 minutes by default. | *`Term Searching Task`*
-Runs (searches and tags) Terms in the workspace
**NOTE:** Each Term Searching run interval will search up to 5 available document batches (default 10,000 documents per batch). These settings are configurable on Term Searching task. | *`Rule Evaluation Task`*
-Evaluates each rule in the workspaces and triggers configured actions |
| Recommended Run Interval | `60` seconds | `300` seconds | `300` seconds | `300` seconds |
-| Considerations and System Impact | -Speed of ingestion is determined by number of Integration Points Agents (max of 4) | -Ongoing index builds use shared instance queue, agents and Fileshare across multiple workspaces (resource pool)
-Every incremental build makes the old build obsolete - cleaned up by Case Manager nightly | | -Very complex/nested underlying Saved Searches can affect performance
-Saved Searches that return many documents can affect performance
-Many rules being evaluated often can put pressure on SQL server |
+| Considerations and System Impact | -Speed of ingestion is determined by number of Integration Points Agents (max of 4 per instance) | -Ongoing index builds use shared instance queue, agents and Fileshare across multiple workspaces (resource pool)
-Every incremental build makes the old build obsolete - cleaned up by Case Manager nightly | | -Very complex/nested underlying Saved Searches can affect performance
-Saved Searches that return many documents can affect performance
-Many rules being evaluated often can put pressure on SQL server |
-> **IMPORTANT**: Reach out to `trace@relativity.com` to adjust the Task `Settings` field
+> **IMPORTANT**: Reach out to `support@relativity.com` to adjust the Task `Settings` field
> **NOTE:** The Recommended Run Interval for the **`Reporting Task`** is 300 seconds.
@@ -1305,18 +1168,16 @@ Appendix A: Trace Object Architecture
Appendix B: Trace Document Extraction Fields
============================================
-Trace automatically extracts metadata information for Microsoft Office 365 Data
-Source and Relativity Data Extraction Data Source. Below is the list of all the
-supported fields. There are four major categories for fields:
+Trace automatically extracts metadata information for Microsoft Office 365 Data Source and Relativity Data Extraction Data Source. Below is the list of all the supported fields. There are four major categories for fields:
-- Email Only – fields that apply to emails only
+- `Email` Only – fields that apply to emails only
-- Email and Document – fields that apply to emails and other documents (Word,
- Excel, PowerPoint, etc.)
+- `Email` and `Document` – fields that apply to emails and other documents (Word,
+ Excel, PowerPoint, etc...)
-- Document Only – fields that apply to documents but not to emails
+- `Document` Only – fields that apply to documents but not to emails
-- Calculated – fields that are calculated dynamically by Trace
+- `Calculated` – fields that are calculated dynamically by Trace
| **Trace Field Category** | **Relativity Field** | **Field Type** | **Field Description** |
|---------------------------|-------------------------------|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
@@ -1391,14 +1252,10 @@ Workspace before proceeding.
> **NOTE:** Load File Templates are generated by Trace Agent **ONLY** in
workspaces that are enabled for “Continuous” run option in “Setup” tab. Ensure `Data Retrieval` task runs **at least once before configuring** Integration Point Profile
-> **NOTE:** Trace uses default Relativity fields for email and attachment data
-that ship with `Create & Map Field Catalog – Full` (v0.0.1.5+). **Trace recommends
-to use this application, however it is totally optional and one can choose to create custom
-fields or re-use fields from an existing template.** You can install
-it to the target workspace like any other application which will bring in all
+> **NOTE:** Trace uses default Relativity fields for email and attachment data that ship with `Create & Map Field Catalog – Full` (v0.0.1.5+). **Trace recommends to use this application, however it is totally optional and one can choose to create custom fields or re-use fields from an existing template.** You can install it to the target workspace like any other application which will bring in all
the needed fields with standardized names that are easily auto-mapped via “Map
Fields” integration point profile feature. If you don’t see this application in
-your library, reach out to `trace@relativity.com`.
+your library, reach out to `support@relativity.com`.
## Setup Integration Point Profile
@@ -1452,6 +1309,4 @@ your library, reach out to `trace@relativity.com`.
5. Press “Save”
-> **NOTE:** You can edit the Integration Point Profile settings at any time,
-however existing completed Data Batches will not automatically re-import the
-data with new settings/mappings.
+> **NOTE:** You can edit the Integration Point Profile settings at any time, however existing completed Data Batches will not automatically re-import the data with new settings/mappings.