Skip to content

Commit

Permalink
[Security Gen AI] [ Integration Assistant ] Modify pipeline template …
Browse files Browse the repository at this point in the history
…to handle dates as array of string in the logs (elastic#187643)

## Summary

There could be
[scenario](https://docs.paloaltonetworks.com/iot/iot-security-api-reference/iot-security-api/get-vulnerability-instances)
where a date can come in as an array -

`"detected_date": [
        "2021-04-19T23:59:59"
        ],`

The `date` processor fails to handle an array with a string.
This PR adds a `script` processor as a pre processor to date processor
to convert the array of string into a string.

Pipeline after the changes

```json
{
      "script": {
        "tag": "script_convert_array_to_string",
        "description": "Ensures the date processor does not receive an array value.",
        "lang": "painless",
        "source": "if (ctx.palo_alto_iot.vulnerability.detected_date instanceof ArrayList){\n    ctx.palo_alto_iot.vulnerability.detected_date = ctx.palo_alto_iot.vulnerability.detected_date[0];\n}\n"
      }
    },
    {
      "date": {
        "if": "ctx.palo_alto_iot?.vulnerability?.detected_date != null",
        "tag": "date_processor_palo_alto_iot.vulnerability.detected_date",
        "field": "palo_alto_iot.vulnerability.detected_date",
        "target_field": "event.start",
        "formats": [
          "ISO8601"
        ]
      }
    }
```

---------

Co-authored-by: Elastic Machine <[email protected]>
  • Loading branch information
bhapas and elasticmachine authored Jul 8, 2024
1 parent ec8475a commit a347d7b
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,19 @@ export const ecsMappingExpectedResults = {
target_field: 'mysql_enterprise.audit',
},
},
{
script: {
description: 'Ensures the date processor does not receive an array value.',
lang: 'painless',
source:
'if (ctx.mysql_enterprise.audit.timestamp instanceof ArrayList){\n ctx.mysql_enterprise.audit.timestamp = ctx.mysql_enterprise.audit.timestamp[0];\n}\n',
tag: 'script_convert_array_to_string',
},
},
{
date: {
field: 'mysql_enterprise.audit.timestamp',
tag: 'date_processor_mysql_enterprise.audit.timestamp',
target_field: '@timestamp',
formats: ['yyyy-MM-dd HH:mm:ss'],
if: 'ctx.mysql_enterprise?.audit?.timestamp != null',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,22 @@ processors:
target_field: {% if value.target_field | startswith('@') %}"{{ value.target_field }}"{% else %}{{ value.target_field }}{% endif %}
ignore_missing: true{% endif %}
{% if key == 'date' %}
- script:
description: Ensures the date processor does not receive an array value.
tag: script_convert_array_to_string
lang: painless
{% raw %}source: |
if (ctx.{% endraw %}{{ value.field }}{% raw %} instanceof ArrayList){
ctx.{% endraw %}{{ value.field }}{% raw %} = ctx.{% endraw %}{{ value.field }}{% raw %}[0];
}{% endraw %}
- {{ key }}:
field: {{ value.field }}
target_field: {% if value.target_field | startswith('@') %}"{{ value.target_field }}"{% else %}{{ value.target_field }}{% endif %}
formats:
{% for format in value.formats %}
- {{ format }}
{% endfor %}
tag: date_processor_{{ value.field}}
if: "ctx.{{ value.if }} != null"{% endif %}
{% if key == 'convert' %}
- {{ key }}:
Expand Down

0 comments on commit a347d7b

Please sign in to comment.