feat: add Graylog Provider

README.md

@@ -115,6 +115,8 @@ Workflow triggers can either be executed manually when an alert is activated or
               
     <img width=32 height=32 src="https://github.com/keephq/keep/blob/main/keep-ui/public/icons/grafana-icon.png?raw=true"/>
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+    <img width=32 height=32 src="https://github.com/keephq/keep/blob/main/keep-ui/public/icons/graylog-icon.png?raw=true"/>
+    &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
     <img width=32 height=32 src="https://github.com/keephq/keep/blob/main/keep-ui/public/icons/prometheus-icon.png?raw=true"/>
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
     <img width=32 height=32 src="https://github.com/keephq/keep/blob/main/keep-ui/public/icons/sumologic-icon.png?raw=true"/>

docs/mint.json

@@ -140,6 +140,7 @@
             "providers/documentation/grafana-provider",
             "providers/documentation/grafana_incident-provider",
             "providers/documentation/grafana_oncall-provider",
+            "providers/documentation/graylog-provider",
             "providers/documentation/http-provider",
             "providers/documentation/ilert-provider",
             "providers/documentation/incidentio-provider",

docs/providers/documentation/graylog-provider.mdx

@@ -0,0 +1,59 @@
+---
+title: "Graylog Provider"
+sidebarTitle: "Graylog Provider"
+description: "The Graylog provider enables webhook installations for receiving alerts in KeepHQ"
+---
+
+## Overview
+
+The **Graylog Provider** facilitates receiving alerts from Graylog by setting up Webhook connections. It allows seamless integration with Graylog to receive notifications about events and alerts through KeepHQ.
+
+## Authentication Parameters
+
+- **Username** (required): Username for authenticating with Graylog's API.
+- **Graylog Access Token** (required): Access token for authenticating with Graylog's API.
+- **Deployment Url** (required): Deployment URL for connecting to the Graylog instance (e.g., `http://localhost:9000`).
+
+## Scopes
+
+- **authenticated**: Mandatory for all operations, ensures the user is authenticated.
+- **authorized**: Mandatory for querying incidents and managing resources, ensures the user has `Admin` privileges.
+
+## Connecting with the Provider
+
+1. Obtain the **username** and **access token** from your Graylog instance by following [Graylog's API Access Documentation](https://go2docs.graylog.org/current/setting_up_graylog/rest_api_access_tokens.htm?tocpath=Set%20up%20Graylog%7CGet%20Started%20with%20Graylog%7CREST%C2%A0API%7C_____3#CreateanAccessToken).
+2. Set the **deployment URL** to your Graylog instance's base URL (e.g., `http://127.0.0.1:9000`).
+3. Ensure the user has the **Admin** role in Graylog.
+
+## Features
+
+The **Graylog Provider** supports the following key features:
+
+- **Webhook Setup**: Configures webhooks to send alerts to KeepHQ.
+- **Alerts Retrieval**: Fetches and formats alerts from Graylog based on specified search parameters (only a maximum of 10000 most recent alerts)
+
+## Inputs for Query
+- **events_search_parameters**: Takes in a python dict
+Example:
+```
+{
+    "filter": {"alerts": "only"},
+    "page": 1,
+    "per_page": 1000,
+    "query": "",
+    "timerange": {"range": 86400, "type": "relative"},
+}
+```
+- You can modify this to fetch either alerts, events or both.
+
+---
+
+**Note**: Ensure that the product of `page` and `per_page` does not exceed 10,000.
+
+---
+
+## Useful Links
+
+- [Graylog API Documentation](https://go2docs.graylog.org/current/what_is_graylog/what_is_graylog.htm?tocpath=What%20Is%20Graylog%253F%7C_____0)
+- [Graylog Access Token](https://go2docs.graylog.org/current/setting_up_graylog/rest_api_access_tokens.htm?tocpath=Set%20up%20Graylog%7CGet%20Started%20with%20Graylog%7CREST%C2%A0API%7C_____3#CreateanAccessToken)
+- [Quick Setup for Graylog & Integration with Keep](https://github.com/keephq/keep/keep/providers/graylog_provider/README.md)

docs/providers/overview.mdx

@@ -268,6 +268,14 @@ By leveraging Keep Providers, users are able to deeply integrate Keep with the t
   }
 ></Card>
 
+<Card
+  title="Graylog"
+  href="/providers/documentation/graylog-provider"
+  icon={
+    <img src="https://img.logo.dev/graylog.com?token=pk_dfXfZBoKQMGDTIgqu7LvYg" />
+  }
+></Card>
+
 <Card
   title="HTTP"
   href="/providers/documentation/http-provider"

keep/providers/graylog_provider/README.md

@@ -0,0 +1,132 @@
+# Instructions for a quick setup
+
+## Setting up Graylog
+
+### Installation
+
+1. Spin up Graylog
+    ```bash
+    cd keep/providers/graylog_provider
+    docker compose up
+    ```
+2. Once the containers are up and running, go to [http://localhost:9000](http://localhost:9000) and sign in with
+   username `admin` & password `admin`.
+
+### Getting Access Token
+
+1. Navigate to System > Users and Teams to view the Users Overview page.
+2. For the user `Admin`, select Edit tokens from the More drop-down menu.
+3. Enter a token name, then click Create Token.
+
+### Setting up Inputs and Event Definition
+
+   ```python
+import requests
+
+auth = ("YOUR_ACCESS_TOKEN", "token")  # from the previous step
+headers = {
+    "Accept": "application/json",
+    "X-Requested-By": "KeepHQ",
+    "Content-Type": "application/json",
+}
+
+input_data = {
+    'type': 'org.graylog2.inputs.raw.tcp.RawTCPInput',
+    'configuration': {
+        'bind_address': '0.0.0.0',
+        'port': 5044,
+        'recv_buffer_size': 1048576,
+        'number_worker_threads': 3,
+        'tls_cert_file': '',
+        'tls_key_file': '',
+        'tls_enable': False,
+        'tls_key_password': '',
+        'tls_client_auth': 'disabled',
+        'tls_client_auth_cert_file': '',
+        'tcp_keepalive': False,
+        'use_null_delimiter': False,
+        'max_message_size': 2097152,
+        'override_source': None,
+        'charset_name': 'UTF-8',
+    },
+    'title': 'KeepHQ-Input',
+    'global': True,
+}
+
+input_response = requests.post(
+    url="http://127.0.0.1:9000/api/system/inputs",
+    headers=headers,
+    json=input_data,
+    auth=auth,
+)
+
+print(input_response.text)
+
+event_data = {
+    'title': 'KeepHQ-Event',
+    'description': 'This is an event for KeepHQ',
+    'priority': 3,
+    'config': {
+        'query': 'source:*',
+        'query_parameters': [],
+        'streams': [],
+        'filters': [],
+        'search_within_ms': 86400000,
+        'execute_every_ms': 60000,
+        'event_limit': 100,
+        'group_by': [],
+        'series': [],
+        'conditions': {},
+        'type': 'aggregation-v1',
+    },
+    'field_spec': {},
+    'key_spec': [],
+    'notification_settings': {
+        'grace_period_ms': 300000,
+        'backlog_size': None,
+    },
+    'notifications': [],
+    'alert': True,
+}
+
+event_response = requests.post(
+    url="http://127.0.0.1:9000/api/events/definitions",
+    headers=headers,
+    json=event_data,
+    auth=auth,
+)
+
+print(event_response.text)
+   ```
+
+### Sending a log
+
+1. After that you can send a plain text message to the Graylog raw/plaintext TCP input running on port 5555 using the
+   following command:
+   ```bash
+   echo 'First log message' | nc localhost 5555
+   ```
+
+## Setup KeepHQ to receive from Graylog
+
+---
+
+### **Note**
+
+1. Run without `NGROK`
+2. After Step 2, do this:
+    - Go to Alerts > Notifications
+    - Click the `title` of the newly create notification > `Edit Notification` > Replace `0.0.0.0` with your ip
+      address > Click `Add to URL whitelist ` > Fill in the `Title` > `Update Configuration` >  `Update Notification`
+
+---
+
+1. Go to `Providers` > search for `Graylog` >
+    - Username: `admin`
+    - Graylog Access Token: Access tokens from previous steps
+    - Deployment Url: http://localhost:9000
+    - Install webhook: True
+
+2. This will create a new notification and install that notification in the existing events.
+3. Send a log to `Graylog`, this will trigger an alert.
+4. Check your feed.

keep/providers/graylog_provider/alerts_mock.py

@@ -0,0 +1,37 @@
+ALERTS = {
+    "event_definition_id": "671a28a03696bb3801a7a9f1",
+    "event_definition_type": "aggregation-v1",
+    "event_definition_title": "Event - 1",
+    "event_definition_description": ".",
+    "job_definition_id": "671a97cc3696bb3801a846a6",
+    "job_trigger_id": "671a9dfe3696bb3801a8536d",
+    "event": {
+        "id": "01JAZZJAKS82TDZAE82E0WAENT",
+        "event_definition_type": "aggregation-v1",
+        "event_definition_id": "671a28a03696bb3801a7a9f1",
+        "origin_context": "urn:graylog:message:es:graylog_0:d0a9a7a0-91f1-11ef-9a79-0242ac170004",
+        "timestamp": "2024-10-24T10:22:04.556Z",
+        "timestamp_processing": "2024-10-24T19:20:30.585Z",
+        "timerange_start": None,
+        "timerange_end": None,
+        "streams": [],
+        "source_streams": ["000000000000000000000001"],
+        "message": "Event - 1",
+        "source": "server",
+        "key_tuple": [],
+        "key": "",
+        "priority": 3,
+        "scores": {},
+        "alert": True,
+        "fields": {},
+        "group_by_fields": {},
+        "replay_info": {
+            "timerange_start": "2024-10-23T19:20:29.706Z",
+            "timerange_end": "2024-10-24T19:20:29.706Z",
+            "query": "source:172.23.0.1",
+            "streams": ["000000000000000000000001"],
+            "filters": [],
+        },
+    },
+    "backlog": [],
+}

keep/providers/graylog_provider/docker-compose.yml

@@ -0,0 +1,102 @@
+version: '3'
+
+services:
+  # MongoDB: https://hub.docker.com/_/mongo/
+  mongodb:
+    image: "mongo:6.0.18"
+    ports:
+      - "27017:27017"
+    restart: "on-failure"
+    networks:
+      - graylog
+    volumes:
+      - "mongodb_data:/data/db"
+
+  opensearch:
+    image: "opensearchproject/opensearch:2.15.0"
+    environment:
+      - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
+      - "bootstrap.memory_lock=true"
+      - "discovery.type=single-node"
+      - "action.auto_create_index=false"
+      - "plugins.security.ssl.http.enabled=false"
+      - "plugins.security.disabled=true"
+      # Can generate a password for `OPENSEARCH_INITIAL_ADMIN_PASSWORD` using a linux device via:
+      # tr -dc A-Z-a-z-0-9_@#%^-_=+ < /dev/urandom | head -c${1:-32}
+      - "OPENSEARCH_INITIAL_ADMIN_PASSWORD=+_8r#wliY3Pv5-HMIf4qzXImYzZf-M=M"
+    ulimits:
+      memlock:
+        hard: -1
+        soft: -1
+      nofile:
+        soft: 65536
+        hard: 65536
+    ports:
+      - "9203:9200"
+      - "9303:9300"
+    restart: "on-failure"
+    networks:
+      - graylog
+    volumes:
+      - "opensearch:/usr/share/opensearch/data"
+
+  # Graylog: https://hub.docker.com/r/graylog/graylog/
+  graylog:
+    hostname: "server"
+    image: "graylog/graylog:6.0"
+    # To install Graylog Open: "graylog/graylog:6.0"
+    depends_on:
+      mongodb:
+        condition: "service_started"
+      opensearch:
+        condition: "service_started"
+    entrypoint: "/usr/bin/tini -- wait-for-it opensearch:9200 -- /docker-entrypoint.sh"
+    environment:
+      GRAYLOG_NODE_ID_FILE: "/usr/share/graylog/data/config/node-id"
+      GRAYLOG_HTTP_BIND_ADDRESS: "0.0.0.0:9000"
+      GRAYLOG_ELASTICSEARCH_HOSTS: "http://opensearch:9200"
+      GRAYLOG_MONGODB_URI: "mongodb://mongodb:27017/graylog"
+      # To make reporting (headless_shell) work inside a Docker container
+      GRAYLOG_REPORT_DISABLE_SANDBOX: "true"
+      # CHANGE ME (must be at least 16 characters)!
+      GRAYLOG_PASSWORD_SECRET: "somepasswordpepper"
+      # Password: "admin"
+      GRAYLOG_ROOT_PASSWORD_SHA2: "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
+      GRAYLOG_HTTP_EXTERNAL_URI: "http://127.0.0.1:9000/"
+    ports:
+      # Graylog web interface and REST API
+      - "9000:9000/tcp"
+      # Beats
+      - "5044:5044/tcp"
+      # Exposing for TCP Ingestion
+      - "5555:5555/tcp"
+      # Syslog TCP
+      - "5140:5140/tcp"
+      # Syslog UDP
+      - "5140:5140/udp"
+      # GELF TCP
+      - "12201:12201/tcp"
+      # GELF UDP
+      - "12201:12201/udp"
+      # Forwarder data
+      - "13301:13301/tcp"
+      # Forwarder config
+      - "13302:13302/tcp"
+    restart: "on-failure"
+    networks:
+      - graylog
+    volumes:
+      - "graylog_data:/usr/share/graylog/data/data"
+      - "graylog_config:/usr/share/graylog/data/config"
+      - "graylog_journal:/usr/share/graylog/data/journal"
+
+networks:
+  graylog:
+    driver: "bridge"
+
+volumes:
+  mongodb_data:
+  opensearch:
+  graylog_data:
+  graylog_config:
+  graylog_journal: