[8.x] [Automatic Import] Add datastream name as dataset value in mani…

…fest (#203106) (#203383)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Automatic Import] Add datastream name as dataset value in manifest
(#203106)](#203106)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Bharat
Pasupula","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-09T10:23:05Z","message":"[Automatic
Import] Add datastream name as dataset value in manifest (#203106)\n\n##
Release note\r\n\r\nUse Data stream name for data_stream.dataset value
in input manifests\r\n\r\n## Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/201478\r\n\r\nThis PR uses the
user-supplied data stream name as the\r\n`data_stream.dataset` value
instead of a generic placeholder.\r\n\r\nCustom Fleet integrations
default to a generic dataset\r\n(data_stream.dataset) value - something
like `http_endpoint.generic.`\r\nSince users configure their own options
for a custom integration, this\r\nworkflow makes sense.\r\n\r\nElastic
integrations, on the other hand, default to a predetermined\r\nnaming
convention based on the integration package + datastream name
for\r\ndataset - something like github.issues for the issues data stream
from\r\nthe Github package.\r\n\r\nHence defaulting it to
`package_name.datastream_name`\r\n\r\n<img width=\"729\"
alt=\"Screenshot 2024-12-05 at 15 12
32\"\r\nsrc=\"https://github.com/user-attachments/assets/2dad5de3-a329-403f-be46-2dbc76906b03\">\r\n\r\n\r\n<img
width=\"868\" alt=\"Screenshot 2024-12-05 at 15 12
16\"\r\nsrc=\"https://github.com/user-attachments/assets/2959e26a-8095-4efa-bf9b-c5e953e849b9\">\r\n\r\n\r\n\r\n###
Checklist\r\n\r\nCheck the PR satisfies following conditions. \r\n\r\n-
[x] The PR description includes the appropriate Release Notes
section,\r\nand the correct `release_note:*` label is applied per
the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"eb1c8fc63f77a166dbf8c9f2485c39f722029e39","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","backport:prev-minor","Team:Security-Scalability","backport:version","Feature:AutomaticImport"],"title":"[Automatic
Import] Add datastream name as dataset value in
manifest","number":203106,"url":"https://github.com/elastic/kibana/pull/203106","mergeCommit":{"message":"[Automatic
Import] Add datastream name as dataset value in manifest (#203106)\n\n##
Release note\r\n\r\nUse Data stream name for data_stream.dataset value
in input manifests\r\n\r\n## Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/201478\r\n\r\nThis PR uses the
user-supplied data stream name as the\r\n`data_stream.dataset` value
instead of a generic placeholder.\r\n\r\nCustom Fleet integrations
default to a generic dataset\r\n(data_stream.dataset) value - something
like `http_endpoint.generic.`\r\nSince users configure their own options
for a custom integration, this\r\nworkflow makes sense.\r\n\r\nElastic
integrations, on the other hand, default to a predetermined\r\nnaming
convention based on the integration package + datastream name
for\r\ndataset - something like github.issues for the issues data stream
from\r\nthe Github package.\r\n\r\nHence defaulting it to
`package_name.datastream_name`\r\n\r\n<img width=\"729\"
alt=\"Screenshot 2024-12-05 at 15 12
32\"\r\nsrc=\"https://github.com/user-attachments/assets/2dad5de3-a329-403f-be46-2dbc76906b03\">\r\n\r\n\r\n<img
width=\"868\" alt=\"Screenshot 2024-12-05 at 15 12
16\"\r\nsrc=\"https://github.com/user-attachments/assets/2959e26a-8095-4efa-bf9b-c5e953e849b9\">\r\n\r\n\r\n\r\n###
Checklist\r\n\r\nCheck the PR satisfies following conditions. \r\n\r\n-
[x] The PR description includes the appropriate Release Notes
section,\r\nand the correct `release_note:*` label is applied per
the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"eb1c8fc63f77a166dbf8c9f2485c39f722029e39"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/203106","number":203106,"mergeCommit":{"message":"[Automatic
Import] Add datastream name as dataset value in manifest (#203106)\n\n##
Release note\r\n\r\nUse Data stream name for data_stream.dataset value
in input manifests\r\n\r\n## Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/201478\r\n\r\nThis PR uses the
user-supplied data stream name as the\r\n`data_stream.dataset` value
instead of a generic placeholder.\r\n\r\nCustom Fleet integrations
default to a generic dataset\r\n(data_stream.dataset) value - something
like `http_endpoint.generic.`\r\nSince users configure their own options
for a custom integration, this\r\nworkflow makes sense.\r\n\r\nElastic
integrations, on the other hand, default to a predetermined\r\nnaming
convention based on the integration package + datastream name
for\r\ndataset - something like github.issues for the issues data stream
from\r\nthe Github package.\r\n\r\nHence defaulting it to
`package_name.datastream_name`\r\n\r\n<img width=\"729\"
alt=\"Screenshot 2024-12-05 at 15 12
32\"\r\nsrc=\"https://github.com/user-attachments/assets/2dad5de3-a329-403f-be46-2dbc76906b03\">\r\n\r\n\r\n<img
width=\"868\" alt=\"Screenshot 2024-12-05 at 15 12
16\"\r\nsrc=\"https://github.com/user-attachments/assets/2959e26a-8095-4efa-bf9b-c5e953e849b9\">\r\n\r\n\r\n\r\n###
Checklist\r\n\r\nCheck the PR satisfies following conditions. \r\n\r\n-
[x] The PR description includes the appropriate Release Notes
section,\r\nand the correct `release_note:*` label is applied per
the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"eb1c8fc63f77a166dbf8c9f2485c39f722029e39"}}]}]
BACKPORT-->

Co-authored-by: Bharat Pasupula <[email protected]>

x-pack/plugins/integration_assistant/server/templates/manifest/cel_manifest.yml.njk

@@ -10,7 +10,8 @@
       title: Dataset name
       description: |
         Dataset to write data to. Changing the dataset will send the data to a different index. You can't use `-` in the name of a dataset and only valid characters for [Elasticsearch index names](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html).
-      default: cel.cel
+      default: |
+        {{ package_name }}.{{ data_stream_name }}
       required: true
       show_user: true
     - name: pipeline

x-pack/plugins/integration_assistant/server/templates/manifest/http_endpoint_manifest.yml.njk

@@ -32,7 +32,8 @@
       title: Dataset name
       description: |
         Dataset to write data to. Changing the dataset will send the data to a different index. You can't use `-` in the name of a dataset and only valid characters for [Elasticsearch index names](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html).
-      default: http_endpoint.generic
+      default: |
+        {{ package_name }}.{{ data_stream_name }}
       required: true
       show_user: true
     - name: pipeline

x-pack/plugins/integration_assistant/server/templates/manifest/kafka_manifest.yml.njk

@@ -26,7 +26,8 @@
       title: Dataset name
       description: |
         Dataset to write data to. Changing the dataset will send the data to a different index. You can't use `-` in the name of a dataset and only valid characters for [Elasticsearch index names](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html).
-      default: kafka_log.generic
+      default: |
+        {{ package_name }}.{{ data_stream_name }}
       required: true
       show_user: true
     - name: pipeline