-
Notifications
You must be signed in to change notification settings - Fork 784
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: GCP logging and friends πββοΈ (#2369)
- Loading branch information
Showing
14 changed files
with
433 additions
and
57 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
117 changes: 76 additions & 41 deletions
117
docs/providers/documentation/gcpmonitoring-provider.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,78 +1,113 @@ | ||
--- | ||
title: "GCP Monitoring" | ||
sidebarTitle: "GCP Monitoring Provider" | ||
description: "GCP Monitoringing provider allows you to get alerts from Azure Monitoring via webhooks." | ||
description: "GCP Monitoring provider allows you to get alerts and logs from GCP Monitoring via webhooks and log queries." | ||
--- | ||
|
||
## Overview | ||
The GCP Monitoring Provider enables seamless integration between Keep and GCP Monitoring, allowing alerts from GCP Monitoring to be directly sent to Keep through webhook configurations. This integration ensures that critical alerts are efficiently managed and responded to within Keep's platform. | ||
|
||
The GCP Monitoring Provider enables seamless integration between Keep and GCP Monitoring, allowing alerts from GCP Monitoring to be directly sent to Keep through webhook configurations. In addition to alerts, the provider now supports querying log entries from GCP Logging, enabling a comprehensive view of alerts and associated logs within Keep's platform. | ||
|
||
## Connecting GCP Monitoring to Keep | ||
To connect GCP Monitoring to Keep, you'll need to configure a webhook as a notification channel in GCP Monitoring and then link it to the desired alert policy. | ||
|
||
### Alert Integration via Webhook | ||
|
||
To connect GCP Monitoring alerts to Keep, configure a webhook as a notification channel in GCP Monitoring and link it to the desired alert policy. | ||
|
||
### Step 1: Access Notification Channels | ||
|
||
Log in to the Google Cloud Platform console. | ||
Navigate to **Monitoring > Alerting > Notification channels**. | ||
|
||
<Frame | ||
width="100" | ||
height="200"> | ||
<img height="10" src="/images/gcpmonitoring_1.png" /> | ||
<Frame width="100" height="200"> | ||
<img height="10" src="/images/gcpmonitoring_1.png" /> | ||
</Frame> | ||
|
||
### Step 2: Add a New Webhook | ||
|
||
Within the Webhooks section, click on **ADD NEW**. | ||
|
||
<Frame | ||
width="100" | ||
height="200"> | ||
<img height="10" src="/images/gcpmonitoring_2.png" /> | ||
<Frame width="100" height="200"> | ||
<img height="10" src="/images/gcpmonitoring_2.png" /> | ||
</Frame> | ||
|
||
### Step 3: Configure the Webhook | ||
|
||
In the Endpoint URL field, enter the webhook URL provided by Keep. | ||
- For Display Name, use keep-gcpmonitoring-webhook-integration. | ||
- Enable Use HTTP Basic Auth and input the following credentials: | ||
- Auth Username: **api_key** | ||
- Auth Password: **%YOURAPIKEY%** | ||
|
||
<Frame | ||
width="100" | ||
height="200"> | ||
<img height="10" src="/images/gcpmonitoring_3.png" /> | ||
|
||
- **Display Name**: keep-gcpmonitoring-webhook-integration | ||
- Enable **Use HTTP Basic Auth** and input the following credentials: | ||
- **Auth Username**: `api_key` | ||
- **Auth Password**: `%YOURAPIKEY%` | ||
|
||
<Frame width="100" height="200"> | ||
<img height="10" src="/images/gcpmonitoring_3.png" /> | ||
</Frame> | ||
|
||
### Step 4: Save the Webhook Configuration | ||
- Click on Save to store the webhook configuration. | ||
|
||
- Click **Save** to store the webhook configuration. | ||
|
||
### Step 5: Associate the Webhook with an Alert Policy | ||
|
||
Navigate to the alert policy you wish to send notifications from to Keep. | ||
- Click on Edit. | ||
- Under "Notifications and name," find the Notification Channels section and select the keep-gcpmonitoring-webhook-integration channel you created. | ||
- Save the changes by clicking on SAVE POLICY. | ||
|
||
<Frame | ||
width="100" | ||
height="200"> | ||
<img height="10" src="/images/gcpmonitoring_4.png" /> | ||
</Frame> | ||
|
||
- Click **Edit**. | ||
- Under "Notifications and name," find the **Notification Channels** section and select the `keep-gcpmonitoring-webhook-integration` channel you created. | ||
- Save the changes by clicking on **SAVE POLICY**. | ||
|
||
<Frame width="100" height="200"> | ||
<img height="10" src="/images/gcpmonitoring_4.png" /> | ||
</Frame> | ||
|
||
<Frame | ||
width="100" | ||
height="200"> | ||
<img height="10" src="/images/gcpmonitoring_5.png" /> | ||
<Frame width="100" height="200"> | ||
<img height="10" src="/images/gcpmonitoring_5.png" /> | ||
</Frame> | ||
|
||
### Step 6: Review the alert in Keep | ||
### Step 6: Review the Alert in Keep | ||
|
||
<Frame | ||
width="100" | ||
height="200"> | ||
<img height="10" src="/images/gcpmonitoring_6.png" /> | ||
Once the setup is complete, alerts from GCP Monitoring will start appearing in Keep. | ||
|
||
<Frame width="100" height="200"> | ||
<img height="10" src="/images/gcpmonitoring_6.png" /> | ||
</Frame> | ||
|
||
## Log Query Integration | ||
|
||
The GCP Monitoring Provider also supports querying logs from GCP Logging, allowing you to fetch log entries based on specific filters. This is helpful for enriching alert data with related logs or for monitoring specific events in Keep. | ||
|
||
### Authentication Requirements | ||
|
||
To enable log querying, you need to provide a service account JSON file with the `logs.viewer` role. This service account should be configured in the `authentication` section of your GCP Monitoring Provider configuration. | ||
|
||
### Querying Logs | ||
|
||
The providerβs `query` function supports filtering logs based on criteria such as resource type, severity, or specific keywords. You can specify a time range for querying logs using `timedelta_in_days`, and control the number of entries with `page_size`. | ||
|
||
#### Example Usage | ||
|
||
Hereβs an example of how you might use the provider to query log entries: | ||
|
||
```python | ||
query(filter='resource.type="cloud_run_revision" AND severity="ERROR"', timedelta_in_days=1) | ||
``` | ||
|
||
This will return logs of severity βERRORβ related to Cloud Run revisions from the past day. | ||
|
||
#### Log Scopes | ||
|
||
To read logs, the provider requires the following IAM role: | ||
|
||
β’ roles/logs.viewer - Allows the provider to read log entries. | ||
|
||
#### Post Installation Validation | ||
|
||
To validate both alerts and logs, follow these steps: | ||
|
||
1. Alert Validation: Test the webhook by triggering an alert in GCP Monitoring and confirm it appears in Keep. | ||
2. Log Query Validation: Execute a simple log query and verify that log entries are returned as expected. | ||
|
||
### Useful Links | ||
- [GCP Monitoring Notification Channels](https://cloud.google.com/monitoring/support/notification-options) | ||
- [GCP Monitoring Alerting](https://cloud.google.com/monitoring/alerts) | ||
|
||
- [GCP Monitoring Notification Channels](https://cloud.google.com/monitoring/support/notification-options) | ||
- [GCP Monitoring Alerting](https://cloud.google.com/monitoring/alerts) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
--- | ||
title: "OpenAI Provider" | ||
description: "The OpenAI Provider allows for integrating OpenAI's language models into Keep." | ||
--- | ||
|
||
<Tip> | ||
The OpenAI Provider supports querying GPT language models for prompt-based | ||
interactions. | ||
</Tip> | ||
|
||
## Inputs | ||
|
||
The OpenAI Provider supports the following functions: | ||
|
||
- `query`: Interact with OpenAI's models by sending prompts and receiving responses | ||
- `model`: The model to be used, defaults to `gpt-3.5-turbo` | ||
|
||
## Outputs | ||
|
||
Currently, the OpenAI Provider outputs the response from the model based on the prompt provided. | ||
|
||
## Authentication Parameters | ||
|
||
To use the OpenAI Provider, you'll need an API Key, and optionally, an Organization ID from OpenAI. The required parameters for authentication are: | ||
|
||
- **api_key** (required): Your OpenAI Platform API Key. | ||
- **organization_id** (optional): Your OpenAI Platform Organization ID. | ||
|
||
## Connecting with the Provider | ||
|
||
To connect to OpenAI, you'll need to obtain an API Key and (optionally) an Organization ID: | ||
|
||
1. Log in to your OpenAI account at [OpenAI Platform](https://platform.openai.com). | ||
2. Go to the **API Keys** section. | ||
3. Click on **Create new secret key** to generate a key for Keep. | ||
4. (Optional) Retrieve your **Organization ID** under **Organization settings** if youβre part of multiple organizations. | ||
|
||
Use the generated API key in the `authentication` section of your OpenAI Provider configuration. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
id: enrich-gcp-alert | ||
description: Enriched GCP Alert | ||
disabled: false | ||
triggers: | ||
- type: manual | ||
- filters: | ||
- key: source | ||
value: gcpmonitoring | ||
type: alert | ||
consts: {} | ||
name: 5a76aa52-4e0f-43c3-85ff-5603229c5d7e | ||
owners: [] | ||
services: [] | ||
steps: | ||
- name: gcpmonitoring-step | ||
provider: | ||
config: "{{ providers.gcp }}" | ||
type: gcpmonitoring | ||
with: | ||
as_json: false | ||
filter: resource.type = "cloud_run_revision" {{alert.traceId}} | ||
page_size: 1000 | ||
raw: false | ||
timedelta_in_days: 1 | ||
- name: openai-step | ||
provider: | ||
config: "{{ providers.openai }}" | ||
type: openai | ||
with: | ||
prompt: | ||
"You are a very talented engineer that receives context from GCP logs | ||
about an endpoint that returned 500 status code and reports back the root | ||
cause analysis. Here is the context: keep.json_dumps({{steps.gcpmonitoring-step.results}}) (it is a JSON list of log entries from GCP Logging). | ||
In your answer, also provide the log entry that made you conclude the root cause and specify what your certainty level is that it is the root cause. (between 1-10, where 1 is low and 10 is high)" | ||
actions: | ||
- name: slack-action | ||
provider: | ||
config: "{{ providers.slack }}" | ||
type: slack | ||
with: | ||
message: "{{steps.openai-step.results}}" |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.