Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new section on how to collect CloudTrail events using Amazon Data Firehose #3823

Merged
merged 23 commits into from
May 7, 2024

Conversation

alaudazzi
Copy link
Contributor

@alaudazzi alaudazzi commented Apr 23, 2024

This tutorial has reached a pretty stable status and is ready to be reviewed.

Doc preview.

Relates to #3819

@alaudazzi alaudazzi added docs Improvements or additions to documentation enhancement New feature or request backport-skip Skip notification from the automated backport with mergify labels Apr 23, 2024
@alaudazzi alaudazzi requested a review from zmoog April 23, 2024 12:42
Copy link
Contributor

A documentation preview will be available soon.

Request a new doc build by commenting
  • Rebuild this PR: run docs-build
  • Rebuild this PR and all Elastic docs: run docs-build rebuild

run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.

If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.

Copy link
Contributor

mergify bot commented Apr 24, 2024

This pull request is now in conflict. Could you fix it @alaudazzi? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b cloudtrail-firehose upstream/cloudtrail-firehose
git merge upstream/main
git push upstream cloudtrail-firehose

@alaudazzi
Copy link
Contributor Author

@zmoog
Thank you for contributing to the procedure with steps and diagrams.
However, with 1ed0a36 the draft from zmoog/public-notes#80 get merged within the current structure but then require a lot of postprocessing to get the right format and a successful build.
Just wondering if this approach might be reconsidered for future doc efforts?

@alaudazzi
Copy link
Contributor Author

To give an example of how image references need to be reformatted

image

@alaudazzi alaudazzi marked this pull request as ready for review May 1, 2024 14:54
@alaudazzi alaudazzi requested a review from a team as a code owner May 1, 2024 14:54
@alaudazzi
Copy link
Contributor Author

alaudazzi commented May 1, 2024

@zmoog
I cleaned up the content and fixed the structure. This PR is ready to be reviewed. I still have some doubts about the readability of the diagrams.


image::firehose-monitor-cloudtrail-logs.png[Firehose monitor CloudTrail logs]

Navigate to {kib} and choose among the following monitoring options:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific use case we could call out with each of these visualization/monitoring options?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are three different ways to visualize the same CloudTrail logs. But I'll defer to @zmoog to confirm.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered one visualization to showcase a detailed view (Discover or Logs Explorer) and one high-level view (dashboard) of CloudTrail logs.

As a user, I often take this two-level approach of getting the feel of the dataset from a quantitative perspective (how many events, etc) and a qualitative perspective (inspecting some sample documents).

Three ways are probably too many.

@bmorelli25 @alaudazzi Is it better to offer one option only, or is there room for two?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Promoting various visualization options is appropriate for tutorial-oriented content. We do the same in the Monitor VPC Flow Logs use case.

@alaudazzi alaudazzi self-assigned this May 2, 2024
alaudazzi and others added 6 commits May 2, 2024 10:54
@zmoog
Copy link
Contributor

zmoog commented May 6, 2024

Just wondering if this approach might be reconsidered for future doc efforts?

Yeah, I see the automation is pretty rough and needs more work.

I may use it personally as an intermediate step, but I'll create a .asciidoc file from now on until the automation improves.

@zmoog
Copy link
Contributor

zmoog commented May 6, 2024

@alaudazzi, I like the changes you made to the initial draft of the issue thread. I'll use this as a reference for the upcoming documents on the AWS Network Firewall and the generic guide on CloudWatch and S3.

@alaudazzi alaudazzi merged commit ec22304 into elastic:main May 7, 2024
3 checks passed
@alaudazzi alaudazzi added backport-8.13 Automated backport with mergify backport-8.14 Automated backport with mergify and removed backport-skip Skip notification from the automated backport with mergify labels May 13, 2024
mergify bot pushed a commit that referenced this pull request May 13, 2024
…ta Firehose (#3823)

* Remove screenshots and adjust text accordingly

* Test the structure

* Add more steps

* Add draft content from public notes

* Fix text and structure 1

* Fix images

* Fix text and structure 2

* Fix text and structure 3

* Fix text and structure 4

* Fix text and structure 5

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Fix headings

* Integrate reviewer's feedback

* Remove link

* Update step 1

---------

Co-authored-by: Maurizio Branca <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
(cherry picked from commit ec22304)

# Conflicts:
#	docs/en/observability/cloud-monitoring/aws/monitor-aws-agent.asciidoc
mergify bot pushed a commit that referenced this pull request May 13, 2024
…ta Firehose (#3823)

* Remove screenshots and adjust text accordingly

* Test the structure

* Add more steps

* Add draft content from public notes

* Fix text and structure 1

* Fix images

* Fix text and structure 2

* Fix text and structure 3

* Fix text and structure 4

* Fix text and structure 5

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Fix headings

* Integrate reviewer's feedback

* Remove link

* Update step 1

---------

Co-authored-by: Maurizio Branca <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
(cherry picked from commit ec22304)

# Conflicts:
#	docs/en/observability/cloud-monitoring/aws/monitor-aws-agent.asciidoc
bmorelli25 added a commit that referenced this pull request May 15, 2024
…ta Firehose (#3823) (#3883)

* Remove screenshots and adjust text accordingly

* Test the structure

* Add more steps

* Add draft content from public notes

* Fix text and structure 1

* Fix images

* Fix text and structure 2

* Fix text and structure 3

* Fix text and structure 4

* Fix text and structure 5

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Fix headings

* Integrate reviewer's feedback

* Remove link

* Update step 1

---------

Co-authored-by: Maurizio Branca <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
(cherry picked from commit ec22304)

# Conflicts:
#	docs/en/observability/cloud-monitoring/aws/monitor-aws-agent.asciidoc

Co-authored-by: Arianna Laudazzi <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
bmorelli25 added a commit that referenced this pull request May 15, 2024
…ta Firehose (#3823) (#3884)

* Remove screenshots and adjust text accordingly

* Test the structure

* Add more steps

* Add draft content from public notes

* Fix text and structure 1

* Fix images

* Fix text and structure 2

* Fix text and structure 3

* Fix text and structure 4

* Fix text and structure 5

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Update docs/en/observability/cloud-monitoring/aws/monitor-aws-cloudtrail-firehose.asciidoc

Co-authored-by: Brandon Morelli <[email protected]>

* Fix headings

* Integrate reviewer's feedback

* Remove link

* Update step 1

---------

Co-authored-by: Maurizio Branca <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
(cherry picked from commit ec22304)

# Conflicts:
#	docs/en/observability/cloud-monitoring/aws/monitor-aws-agent.asciidoc

Co-authored-by: Arianna Laudazzi <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.13 Automated backport with mergify backport-8.14 Automated backport with mergify docs Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants