Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x](backport #41188) [Filebeat] [AWS] Add support to source AWS cloudwatch logs from linked accounts #41240

Merged
merged 2 commits into from
Oct 15, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Oct 15, 2024

Proposed commit message

PR adds support to Cloudwatch logs from source linked accounts. This is implemented by using existing configuration log_group_arn and mapping it to LogGroupIdentifier of FilterLogEvents API [1]

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Note for reviewers

You could review commit by commit for better understanding of the changes

How to test this PR locally

This require a linked cloudwatch account. If already has one, then,

  • Push logs to a newly created log group OR use an already existing log group in a source account
    • Note - you may use data-gen Go program to generate and push logs to your log group (using output CLOUDWATCH_LOG) [2]
  • Configure filebeat cloudwatch input with log group ARN to log_group_arn
  • Run filebeat and observe filebeat logs in Kibana discover

Related issues

Addresses: #36642
And makes #36645 PR obsolete
closes #37681

Next step

Utilize includeLinkedAccounts when dealing with prefixes. To be done in a dedicated PR.

[1] - https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_FilterLogEvents.html
[1] - https://github.com/Kavindu-Dodan/data-gen


This is an automatic backport of pull request #41188 done by Mergify.

…d accounts (#41188)

* use LogGroupIdentifier fiter instead of LogGroupName and related parameter, field renaming

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* configuration parsing to support arn & linked accounts

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* document the ARN usage

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* add changelog entry

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* code review changes

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* code review change - fix typo

Signed-off-by: Kavindu Dodanduwa <[email protected]>

---------

Signed-off-by: Kavindu Dodanduwa <[email protected]>
Co-authored-by: kaiyan-sheng <[email protected]>
(cherry picked from commit 42f2d41)
@mergify mergify bot requested review from a team as code owners October 15, 2024 14:24
@mergify mergify bot added the backport label Oct 15, 2024
@mergify mergify bot removed the request for review from a team October 15, 2024 14:24
@mergify mergify bot requested review from belimawr and mauri870 October 15, 2024 14:24
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 15, 2024
@Kavindu-Dodan Kavindu-Dodan added the Team:obs-ds-hosted-services Label for the Observability Hosted Services team label Oct 15, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 15, 2024
Signed-off-by: Kavindu Dodanduwa <[email protected]>
@Kavindu-Dodan Kavindu-Dodan merged commit 048a8ea into 8.x Oct 15, 2024
22 checks passed
@Kavindu-Dodan Kavindu-Dodan deleted the mergify/bp/8.x/pr-41188 branch October 15, 2024 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:obs-ds-hosted-services Label for the Observability Hosted Services team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants