[ti_anomali,ti_recordedfuture] Add destination index alias and update…

… docs. (#9618)

* Anomali, RecordedFuture: add destination index alias and update docs.

packages/ti_anomali/_dev/build/docs/README.md

@@ -17,7 +17,7 @@ explanation on how to configure the Anomali ThreatStream to send indicator
 to this integration.
 
 ### Expiration of Indicators of Compromise (IOCs)
-The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_anomali_latest.threatstream` which only contains active and unexpired IOCs. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_anomali.threatstream-*` indices.
+The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_anomali_latest.threatstream-2` which only contains active and unexpired IOCs. The destination index also has an alias `logs-ti_anomali_latest.threatstream`. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_anomali.threatstream-*` indices.
 
 #### Handling Orphaned IOCs
 When an IOC expires, Anomali feed contains information about all IOCs that got `deleted`. However, some Anomali IOCs may never expire and will continue to stay in the latest destination index `logs-ti_anomali_latest.threatstream`. To avoid any false positives from such orphaned IOCs, users are allowed to configure `IOC Expiration Duration` parameter while setting up the integration. This parameter deletes all data inside the destination index `logs-ti_anomali_latest.threatstream` after this specified duration is reached. Users must pull entire feed instead of incremental feed when this expiration happens so that the IOCs get reset. 

packages/ti_anomali/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.21.0"
+  changes:
+    - description: Add destination index alias and fix docs.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/9618
 - version: "1.20.0"
   changes:
     - description: Set sensitive values as secret.

packages/ti_anomali/docs/README.md

@@ -17,7 +17,7 @@ explanation on how to configure the Anomali ThreatStream to send indicator
 to this integration.
 
 ### Expiration of Indicators of Compromise (IOCs)
-The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_anomali_latest.threatstream` which only contains active and unexpired IOCs. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_anomali.threatstream-*` indices.
+The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_anomali_latest.threatstream-2` which only contains active and unexpired IOCs. The destination index also has an alias `logs-ti_anomali_latest.threatstream`. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_anomali.threatstream-*` indices.
 
 #### Handling Orphaned IOCs
 When an IOC expires, Anomali feed contains information about all IOCs that got `deleted`. However, some Anomali IOCs may never expire and will continue to stay in the latest destination index `logs-ti_anomali_latest.threatstream`. To avoid any false positives from such orphaned IOCs, users are allowed to configure `IOC Expiration Duration` parameter while setting up the integration. This parameter deletes all data inside the destination index `logs-ti_anomali_latest.threatstream` after this specified duration is reached. Users must pull entire feed instead of incremental feed when this expiration happens so that the IOCs get reset. 

packages/ti_anomali/elasticsearch/transform/latest_ioc/transform.yml

@@ -10,6 +10,9 @@ source:
 # time field type conflicts.
 dest:
   index: "logs-ti_anomali_latest.threatstream-2"
+  aliases:
+    - alias: "logs-ti_anomali_latest.threatstream"
+      move_on_creation: true
 latest:
   unique_key:
     - event.dataset
@@ -29,4 +32,4 @@ retention_policy:
 _meta:
   managed: true
   # Bump this version to delete, reinstall, and restart the transform during package.
-  fleet_transform_version: 0.2.0
+  fleet_transform_version: 0.3.0

packages/ti_anomali/manifest.yml

@@ -1,6 +1,6 @@
 name: ti_anomali
 title: Anomali
-version: "1.20.0"
+version: "1.21.0"
 description: Ingest threat intelligence indicators from Anomali with Elastic Agent.
 type: integration
 format_version: 3.0.2

packages/ti_recordedfuture/_dev/build/docs/README.md

@@ -11,7 +11,7 @@ Alternatively, it's also possible to use the integration to fetch custom Fusion
 by supplying the URL to the CSV file as the _Custom_ _URL_ configuration option.
 
 ### Expiration of Indicators of Compromise (IOCs)
-The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_recordedfuture_latest.threat` which only contains active and unexpired IOCs. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_recordedfuture.threat-*` indices.
+The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_recordedfuture_latest.threat-1` which only contains active and unexpired IOCs. The destination index also has an alias `logs-ti_recordedfuture_latest.threat`. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_recordedfuture.threat-*` indices.
 
 ### ILM Policy
 To facilitate IOC expiration, source datastream-backed indices `.ds-logs-ti_recordedfuture.threat-*` are allowed to contain duplicates from each polling interval. ILM policy is added to these source indices so it doesn't lead to unbounded growth. This means data in these source indices will be deleted after `5 days` from ingested date. 

packages/ti_recordedfuture/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.24.0"
+  changes:
+    - description: Add destination index alias and fix docs.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/9618
 - version: "1.23.0"
   changes:
     - description: Add dashboards and list field

packages/ti_recordedfuture/docs/README.md

@@ -11,7 +11,7 @@ Alternatively, it's also possible to use the integration to fetch custom Fusion
 by supplying the URL to the CSV file as the _Custom_ _URL_ configuration option.
 
 ### Expiration of Indicators of Compromise (IOCs)
-The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_recordedfuture_latest.threat` which only contains active and unexpired IOCs. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_recordedfuture.threat-*` indices.
+The ingested IOCs expire after certain duration. An [Elastic Transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named `logs-ti_recordedfuture_latest.threat-1` which only contains active and unexpired IOCs. The destination index also has an alias `logs-ti_recordedfuture_latest.threat`. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read [ILM Policy](#ilm-policy) below which is added to avoid unbounded growth on source `.ds-logs-ti_recordedfuture.threat-*` indices.
 
 ### ILM Policy
 To facilitate IOC expiration, source datastream-backed indices `.ds-logs-ti_recordedfuture.threat-*` are allowed to contain duplicates from each polling interval. ILM policy is added to these source indices so it doesn't lead to unbounded growth. This means data in these source indices will be deleted after `5 days` from ingested date. 

packages/ti_recordedfuture/elasticsearch/transform/latest_ioc/transform.yml

@@ -10,6 +10,9 @@ source:
 # time field type conflicts.
 dest:
   index: "logs-ti_recordedfuture_latest.threat-1"
+  aliases:
+    - alias: "logs-ti_recordedfuture_latest.threat"
+      move_on_creation: true
 latest:
   unique_key:
     - event.dataset
@@ -32,4 +35,4 @@ _meta:
   managed: true
   # Bump this version to delete, reinstall, and restart the transform during package.
   # Version bump is needed if there is any code change in transform.
-  fleet_transform_version: 0.1.0
+  fleet_transform_version: 0.2.0

packages/ti_recordedfuture/manifest.yml

@@ -1,6 +1,6 @@
 name: ti_recordedfuture
 title: Recorded Future
-version: "1.23.0"
+version: "1.24.0"
 description: Ingest threat intelligence indicators from Recorded Future risk lists with Elastic Agent.
 type: integration
 format_version: 3.0.2