From 586f0fbb6b8ba0de4baa0408970a952622eb6b35 Mon Sep 17 00:00:00 2001 From: Michael Wolf Date: Fri, 6 Dec 2024 16:52:09 -0800 Subject: [PATCH] [checkpoint] Update README with instructions on using logfile input (#11766) Revises the Check Point integration readme to include detailed instructions on utilizing the logfile input feature. --- packages/checkpoint/_dev/build/docs/README.md | 39 +- packages/checkpoint/changelog.yml | 5 + packages/checkpoint/changelog.yml.orig | 364 ++++++++++++++++++ packages/checkpoint/docs/README.md | 39 +- packages/checkpoint/manifest.yml | 2 +- 5 files changed, 428 insertions(+), 21 deletions(-) create mode 100644 packages/checkpoint/changelog.yml.orig diff --git a/packages/checkpoint/_dev/build/docs/README.md b/packages/checkpoint/_dev/build/docs/README.md index 127c3199a89..95cd4315557 100644 --- a/packages/checkpoint/_dev/build/docs/README.md +++ b/packages/checkpoint/_dev/build/docs/README.md @@ -26,21 +26,40 @@ This integration has been tested against Check Point Log Exporter on R81.X. ## Setup -1. Install Elastic Agent on a host between your Check Point Log Exporter instance and Elastic Cluster. The agent will be used to receive syslog data from your Check Point firewalls and ship the events to Elasticsearch. -2. For each firewall device you wish to monitor, create a new [Log Exporter/SIEM object](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Configuration-in-SmartConsole.htm?tocpath=Log%20Exporter%7C_____2) in Check Point *SmartConsole*. Set the target server and target port to the Elastic Agent IP address and port number. Set the protocol to UDP or TCP, the Check Point integration supports both. Set the format to syslog. -3. Configure the Management Server or Dedicated Log Server object in *SmartConsole*. -4. Install the database within *SmartConsole* (steps included in the Checkpoint docs linked above). -5. Within Kibana, browse to Integrations and locate the Check Point integration, and 'Add Check Point' -6. Configure the TCP or UDP input, depending on the protocol you configured Check Point to use. -7. Add a certificate if using Secure Syslog over TCP with TLS (optional) -8. Add integration to a New/Existing policy. -9. Browse to dashboard/discover to validate data is flowing from Check Point. - For step-by-step instructions on how to set up an integration, see the [Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide. In some instances firewall events may have the same Checkpoint `loguid` and arrive during the same timestamp resulting in a fingerprint collision. To avoid this [enable semi-unified logging](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Appendix.htm?TocPath=Log%20Exporter%7C_____9) in the Checkpoint dashboard. +### TCP or UDP + +Elastic Agent can receive log messages directly via TCP or UDP syslog messages. The Elastic Agent will be used to receive syslog data from your Check Point firewalls and ship the events to Elasticsearch. + +1. For each firewall device you wish to monitor, create a new [Log Exporter/SIEM object](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Configuration-in-SmartConsole.htm?tocpath=Log%20Exporter%7C_____2) in Check Point *SmartConsole*. Set the target server and target port to the Elastic Agent IP address and port number. Set the protocol to UDP or TCP, the Check Point integration supports both. Set the format to syslog. +2. Configure the Management Server or Dedicated Log Server object in *SmartConsole*. +3. Install the database within *SmartConsole* (steps included in the Checkpoint docs linked above). +4. Within Kibana, browse to Integrations and locate the Check Point integration, and 'Add Check Point'. +5. Add Elastic Agent to host with Fleet, or install Elastic Agent manually after configuring the integration. +6. Configure the TCP or UDP input, depending on the protocol you configured Check Point to use. +7. Add a certificate if using Secure Syslog over TCP with TLS (optional) +8. Add integration to a New/Existing policy. +9. Browse to dashboard/discover to validate data is flowing from Check Point. + +### Logfile + +Elastic Agent can process log messages by monitoring a log file on a host receiving syslog messages. The syslog server will receive messages from Check Point, write to a logfile, and Elastic Agent will watch the log file to send to the Elastic Cluster. + +1. Install a syslog server on a host between your Check Point Log Exporter instance and Elastic Cluster. +2. Configure the syslog server to write logs to a logfile. +3. For each firewall device you wish to monitor, create a new [Log Exporter/SIEM object](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Configuration-in-SmartConsole.htm?tocpath=Log%20Exporter%7C_____2) in Check Point *SmartConsole*. Set the target server and target port to the syslog server. Set the protocol to UDP or TCP, the Check Point integration supports both. Set the format to syslog. +4. Configure the Management Server or Dedicated Log Server object in *SmartConsole*. +5. Install the database within *SmartConsole* (steps included in the Checkpoint docs linked above). +6. Within Kibana, navigate to the Integrations section and locate the Check Point integration. Click on the "Add Check Point" button to initiate the integration process. +7. Add Elastic Agent to host with Fleet, or install Elastic Agent manually after configuring the integration. +8. Configure the logfile input, to monitor the logfile pattern that the syslog server will write to. +9. Add integration to a New/Existing policy. +10. Browse to dashboard/discover to validate data is flowing from Check Point. + ## Logs reference ### Firewall diff --git a/packages/checkpoint/changelog.yml b/packages/checkpoint/changelog.yml index 8f6879b5394..9c7f853eea0 100644 --- a/packages/checkpoint/changelog.yml +++ b/packages/checkpoint/changelog.yml @@ -1,4 +1,9 @@ # newer versions go on top +- version: "1.34.4" + changes: + - description: Add instructions on using logfile input + type: bugfix + link: https://github.com/elastic/integrations/pull/11766 - version: "1.34.3" changes: - description: Align hostname grok pattern with syslog RFC. diff --git a/packages/checkpoint/changelog.yml.orig b/packages/checkpoint/changelog.yml.orig new file mode 100644 index 00000000000..1f4a190e85e --- /dev/null +++ b/packages/checkpoint/changelog.yml.orig @@ -0,0 +1,364 @@ +# newer versions go on top +- version: "1.34.3" + changes: +<<<<<<< HEAD + - description: Add instructions on using logfile input + type: bugfix + link: https://github.com/elastic/integrations/pull/11766 +======= + - description: Align hostname grok pattern with syslog RFC. + type: bugfix + link: https://github.com/elastic/integrations/pull/11947 +>>>>>>> upstream/main +- version: "1.34.2" + changes: + - description: Use triple-brace Mustache templating when referencing variables in ingest pipelines. + type: bugfix + link: https://github.com/elastic/integrations/pull/11315 +- version: "1.34.1" + changes: + - description: Use triple-brace Mustache templating when referencing variables in ingest pipelines. + type: bugfix + link: https://github.com/elastic/integrations/pull/11286 +- version: "1.34.0" + changes: + - description: Drop support for EOL OS version R80.X + type: enhancement + link: https://github.com/elastic/integrations/pull/11263 +- version: "1.33.1" + changes: + - description: Improve normalization of user.name field + type: enhancement + link: https://github.com/elastic/integrations/pull/10896 +- version: "1.33.0" + changes: + - description: "Allow @custom pipeline access to event.original without setting preserve_original_event." + type: enhancement + link: https://github.com/elastic/integrations/pull/10897 +- version: "1.32.0" + changes: + - description: Migrate log stream visualization to saved search. + type: enhancement + link: https://github.com/elastic/integrations/pull/10815 +- version: "1.31.1" + changes: + - description: Ensure event.original is always set to value of message. + type: bugfix + link: https://github.com/elastic/integrations/pull/10645 +- version: "1.31.0" + changes: + - description: Update package-spec to 3.0.3. + type: enhancement + link: https://github.com/elastic/integrations/pull/9235 +- version: "1.30.2" + changes: + - description: Changed owners + type: enhancement + link: https://github.com/elastic/integrations/pull/8943 +- version: 1.30.1 + changes: + - description: Add missing fields for audit log events. + type: bugfix + link: https://github.com/elastic/integrations/pull/8910 +- version: 1.30.0 + changes: + - description: Improve authentication logs normalization. + type: enhancement + link: https://github.com/elastic/integrations/pull/8884 +- version: "1.29.1" + changes: + - description: Fix exclude_files pattern. + type: bugfix + link: https://github.com/elastic/integrations/pull/8635 +- version: 1.29.0 + changes: + - description: ECS version updated to 8.11.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/8433 +- version: "1.28.0" + changes: + - description: Improve 'event.original' check to avoid errors if set. + type: enhancement + link: https://github.com/elastic/integrations/pull/8269 +- version: 1.27.0 + changes: + - description: ECS version updated to 8.10.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/7905 +- version: 1.26.0 + changes: + - description: "The format_version in the package manifest changed from 2.11.0 to 3.0.0. Removed dotted YAML keys from package manifest. Added 'owner.type: elastic' to package manifest." + type: enhancement + link: https://github.com/elastic/integrations/pull/7883 +- version: "1.25.0" + changes: + - description: Add tags.yml file so that integration's dashboards and saved searches are tagged with "Security Solution" and displayed in the Security Solution UI. + type: enhancement + link: https://github.com/elastic/integrations/pull/7789 +- version: "1.24.0" + changes: + - description: Ensure `checkpoint.subs_exp` is a date. + type: enhancement + link: https://github.com/elastic/integrations/pull/7714 +- version: "1.23.0" + changes: + - description: Update package to ECS 8.9.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/7107 +- version: "1.22.0" + changes: + - description: Avoid data loss from updates with colliding loguid and timestamp. + type: enhancement + link: https://github.com/elastic/integrations/pull/6483 +- version: "1.21.0" + changes: + - description: Update package to ECS 8.8.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/6325 +- version: "1.20.0" + changes: + - description: Update package-spec version to 2.7.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/6135 +- version: "1.19.0" + changes: + - description: Update package to ECS 8.7.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/5765 +- version: "1.18.0" + changes: + - description: Improve documentation. + type: enhancement + link: https://github.com/elastic/integrations/pull/5627 +- version: "1.17.0" + changes: + - description: Add dashboards. + type: enhancement + link: https://github.com/elastic/integrations/pull/5472 +- version: "1.16.1" + changes: + - description: Added categories and/or subcategories. + type: enhancement + link: https://github.com/elastic/integrations/pull/5123 +- version: "1.16.0" + changes: + - description: Add support for new R81 fields. + type: enhancement + link: https://github.com/elastic/integrations/pull/5440 + - description: Enhance error handling. + type: enhancement + link: https://github.com/elastic/integrations/pull/5440 + - description: Support logs with multiple time values. + type: enhancement + link: https://github.com/elastic/integrations/pull/5440 + - description: Fingerprint events to prevent duplicate ingestion. + type: enhancement + link: https://github.com/elastic/integrations/pull/5440 +- version: "1.15.1" + changes: + - description: Fix Check Point `src_user_name` field mapping. + type: bugfix + link: https://github.com/elastic/integrations/pull/5321 +- version: "1.15.0" + changes: + - description: Enhance error handling. + type: enhancement + link: https://github.com/elastic/integrations/pull/5295 +- version: "1.14.0" + changes: + - description: Expose `origin_sic_name` field. + type: enhancement + link: https://github.com/elastic/integrations/pull/5220 + - description: Improve structured data handling. + type: enhancement + link: https://github.com/elastic/integrations/pull/5220 +- version: "1.13.0" + changes: + - description: Improve support for Checkpoint 81. + type: enhancement + link: https://github.com/elastic/integrations/pull/5184 +- version: "1.12.0" + changes: + - description: Allow configuration of time zones. + type: enhancement + link: https://github.com/elastic/integrations/pull/5157 +- version: "1.11.0" + changes: + - description: Update package to ECS 8.6.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/4576 +- version: "1.10.0" + changes: + - description: Add `udp_options` to the UDP input. + type: enhancement + link: https://github.com/elastic/integrations/pull/4863 +- version: "1.9.1" + changes: + - description: Support `checkpoint.time` field as both UNIX and UNIX_MS + type: bugfix + link: https://github.com/elastic/integrations/pull/4781 +- version: "1.9.0" + changes: + - description: Update package to ECS 8.5.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/4285 +- version: "1.8.2" + changes: + - description: Remove duplicate field. + type: enhancement + link: https://github.com/elastic/integrations/issues/4339 +- version: "1.8.1" + changes: + - description: Use ECS geo.location definition. + type: enhancement + link: https://github.com/elastic/integrations/issues/4227 +- version: "1.8.0" + changes: + - description: Update package to ECS 8.4.0 + type: enhancement + link: https://github.com/elastic/integrations/pull/3842 +- version: "1.7.1" + changes: + - description: Fix handling of R81 fields. + type: bugfix + link: https://github.com/elastic/integrations/pull/3800 +- version: "1.7.0" + changes: + - description: Add handling of authentication events. + type: enhancement + link: https://github.com/elastic/integrations/pull/3750 +- version: "1.6.1" + changes: + - description: Improve TCP, SSL config description and example. + type: enhancement + link: https://github.com/elastic/integrations/pull/3763 +- version: "1.6.0" + changes: + - description: Update package to ECS 8.3.0. + type: enhancement + link: https://github.com/elastic/integrations/pull/3353 +- version: "1.5.1" + changes: + - description: Update Checkpoint logo. + type: enhancement + link: https://github.com/elastic/integrations/pull/3557 +- version: "1.5.0" + changes: + - description: Add TLS and custom options support to TCP input. + type: enhancement + link: https://github.com/elastic/integrations/pull/3317 +- version: "1.4.0" + changes: + - description: Update to ECS 8.2 to use new email field set. + type: enhancement + link: https://github.com/elastic/integrations/pull/2803 +- version: "1.3.6" + changes: + - description: Fixed parsing error when logs have trailing spaces + type: bugfix + link: https://github.com/elastic/integrations/pull/3035 +- version: "1.3.5" + changes: + - description: Added link to check point documentation. + type: enhancement + link: https://github.com/elastic/integrations/pull/2926 +- version: "1.3.4" + changes: + - description: Change mapping type of checkpoint.source_object to keyword from integer. + type: bugfix + link: https://github.com/elastic/integrations/pull/2951 +- version: "1.3.3" + changes: + - description: Add documentation for multi-fields + type: enhancement + link: https://github.com/elastic/integrations/pull/2916 +- version: "1.3.2" + changes: + - description: Fix field mapping conflicts for `checkpoint.icmp_type`, `checkpoint.icmp_code` & `checkpoint.email_recipients_num` + type: bugfix + link: https://github.com/elastic/integrations/pull/2895 +- version: "1.3.1" + changes: + - description: Add Ingest Pipeline script to map IANA Protocol Numbers + type: bugfix + link: https://github.com/elastic/integrations/pull/2470 +- version: "1.3.0" + changes: + - description: Update to ECS 8.0 + type: enhancement + link: https://github.com/elastic/integrations/pull/2387 +- version: "1.2.2" + changes: + - description: Regenerate test files using the new GeoIP database + type: bugfix + link: https://github.com/elastic/integrations/pull/2339 +- version: "1.2.1" + changes: + - description: Change test public IPs to the supported subset + type: bugfix + link: https://github.com/elastic/integrations/pull/2327 +- version: "1.2.0" + changes: + - description: Add 8.0.0 version constraint + type: enhancement + link: https://github.com/elastic/integrations/pull/2231 +- version: "1.1.2" + changes: + - description: Update Title and Description. + type: enhancement + link: https://github.com/elastic/integrations/pull/1951 +- version: "1.1.1" + changes: + - description: Fix logic that checks for the 'forwarded' tag + type: bugfix + link: https://github.com/elastic/integrations/pull/1803 +- version: "1.1.0" + changes: + - description: Update to ECS 1.12.0 + type: enhancement + link: https://github.com/elastic/integrations/pull/1653 +- version: "1.0.0" + changes: + - description: make GA + type: enhancement + link: https://github.com/elastic/integrations/pull/1605 +- version: "0.8.2" + changes: + - description: Convert to generated ECS fields + type: enhancement + link: https://github.com/elastic/integrations/pull/1470 +- version: '0.8.1' + changes: + - description: update to ECS 1.11.0 + type: enhancement + link: https://github.com/elastic/integrations/pull/1376 +- version: "0.8.0" + changes: + - description: Update integration description + type: enhancement + link: https://github.com/elastic/integrations/pull/1364 +- version: "0.7.0" + changes: + - description: Set "event.module" and "event.dataset" + type: enhancement + link: https://github.com/elastic/integrations/pull/1256 +- version: "0.6.0" + changes: + - description: update to ECS 1.10.0 and syncing module changes + type: enhancement + link: https://github.com/elastic/integrations/pull/1033 +- version: "0.5.2" + changes: + - description: update to ECS 1.9.0 + type: enhancement + link: https://github.com/elastic/integrations/pull/839 +- version: "0.5.1" + changes: + - description: Change kibana.version constraint to be more conservative. + type: bugfix + link: https://github.com/elastic/integrations/pull/749 +- version: "0.1.0" + changes: + - description: initial release + type: enhancement # can be one of: enhancement, bugfix, breaking-change + link: https://github.com/elastic/integrations/pull/220 diff --git a/packages/checkpoint/docs/README.md b/packages/checkpoint/docs/README.md index ae0156127eb..9fb47ee2437 100644 --- a/packages/checkpoint/docs/README.md +++ b/packages/checkpoint/docs/README.md @@ -26,21 +26,40 @@ This integration has been tested against Check Point Log Exporter on R81.X. ## Setup -1. Install Elastic Agent on a host between your Check Point Log Exporter instance and Elastic Cluster. The agent will be used to receive syslog data from your Check Point firewalls and ship the events to Elasticsearch. -2. For each firewall device you wish to monitor, create a new [Log Exporter/SIEM object](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Configuration-in-SmartConsole.htm?tocpath=Log%20Exporter%7C_____2) in Check Point *SmartConsole*. Set the target server and target port to the Elastic Agent IP address and port number. Set the protocol to UDP or TCP, the Check Point integration supports both. Set the format to syslog. -3. Configure the Management Server or Dedicated Log Server object in *SmartConsole*. -4. Install the database within *SmartConsole* (steps included in the Checkpoint docs linked above). -5. Within Kibana, browse to Integrations and locate the Check Point integration, and 'Add Check Point' -6. Configure the TCP or UDP input, depending on the protocol you configured Check Point to use. -7. Add a certificate if using Secure Syslog over TCP with TLS (optional) -8. Add integration to a New/Existing policy. -9. Browse to dashboard/discover to validate data is flowing from Check Point. - For step-by-step instructions on how to set up an integration, see the [Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide. In some instances firewall events may have the same Checkpoint `loguid` and arrive during the same timestamp resulting in a fingerprint collision. To avoid this [enable semi-unified logging](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Appendix.htm?TocPath=Log%20Exporter%7C_____9) in the Checkpoint dashboard. +### TCP or UDP + +Elastic Agent can receive log messages directly via TCP or UDP syslog messages. The Elastic Agent will be used to receive syslog data from your Check Point firewalls and ship the events to Elasticsearch. + +1. For each firewall device you wish to monitor, create a new [Log Exporter/SIEM object](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Configuration-in-SmartConsole.htm?tocpath=Log%20Exporter%7C_____2) in Check Point *SmartConsole*. Set the target server and target port to the Elastic Agent IP address and port number. Set the protocol to UDP or TCP, the Check Point integration supports both. Set the format to syslog. +2. Configure the Management Server or Dedicated Log Server object in *SmartConsole*. +3. Install the database within *SmartConsole* (steps included in the Checkpoint docs linked above). +4. Within Kibana, browse to Integrations and locate the Check Point integration, and 'Add Check Point'. +5. Add Elastic Agent to host with Fleet, or install Elastic Agent manually after configuring the integration. +6. Configure the TCP or UDP input, depending on the protocol you configured Check Point to use. +7. Add a certificate if using Secure Syslog over TCP with TLS (optional) +8. Add integration to a New/Existing policy. +9. Browse to dashboard/discover to validate data is flowing from Check Point. + +### Logfile + +Elastic Agent can process log messages by monitoring a log file on a host receiving syslog messages. The syslog server will receive messages from Check Point, write to a logfile, and Elastic Agent will watch the log file to send to the Elastic Cluster. + +1. Install a syslog server on a host between your Check Point Log Exporter instance and Elastic Cluster. +2. Configure the syslog server to write logs to a logfile. +3. For each firewall device you wish to monitor, create a new [Log Exporter/SIEM object](https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-Configuration-in-SmartConsole.htm?tocpath=Log%20Exporter%7C_____2) in Check Point *SmartConsole*. Set the target server and target port to the syslog server. Set the protocol to UDP or TCP, the Check Point integration supports both. Set the format to syslog. +4. Configure the Management Server or Dedicated Log Server object in *SmartConsole*. +5. Install the database within *SmartConsole* (steps included in the Checkpoint docs linked above). +6. Within Kibana, navigate to the Integrations section and locate the Check Point integration. Click on the "Add Check Point" button to initiate the integration process. +7. Add Elastic Agent to host with Fleet, or install Elastic Agent manually after configuring the integration. +8. Configure the logfile input, to monitor the logfile pattern that the syslog server will write to. +9. Add integration to a New/Existing policy. +10. Browse to dashboard/discover to validate data is flowing from Check Point. + ## Logs reference ### Firewall diff --git a/packages/checkpoint/manifest.yml b/packages/checkpoint/manifest.yml index 2fd9ac60d02..6f663246003 100644 --- a/packages/checkpoint/manifest.yml +++ b/packages/checkpoint/manifest.yml @@ -1,6 +1,6 @@ name: checkpoint title: Check Point -version: "1.34.3" +version: "1.34.4" description: Collect logs from Check Point with Elastic Agent. type: integration format_version: "3.0.3"