Merge branch 'main' into feat-2538-refactor-builder-actions

Signed-off-by: Rajesh Jonnalagadda <[email protected]>

.gitignore

@@ -214,3 +214,8 @@ oauth2.cfg
 scripts/keep_slack_bot.py
 keepnew.db
 providers_cache.json
+
+tests/provision/*
+grafana/*
+!grafana/provisioning/
+!grafana/dashboards/

docker-compose.yml

@@ -26,3 +26,33 @@ services:
     extends:
       file: docker-compose.common.yml
       service: keep-websocket-server-common
+
+  grafana:
+    image: grafana/grafana:latest
+    profiles:
+      - grafana
+    ports:
+      - "3001:3000"
+    volumes:
+      - ./grafana:/var/lib/grafana
+      - ./grafana/provisioning:/etc/grafana/provisioning
+      - ./grafana/dashboards:/etc/grafana/dashboards
+    environment:
+      - GF_SECURITY_ADMIN_USER=admin
+      - GF_SECURITY_ADMIN_PASSWORD=admin
+      - GF_USERS_ALLOW_SIGN_UP=false
+    depends_on:
+      - prometheus
+
+  prometheus:
+    image: prom/prometheus:latest
+    profiles:
+      - grafana
+    ports:
+      - "9090:9090"
+    volumes:
+      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
+    command:
+      - "--config.file=/etc/prometheus/prometheus.yml"
+    depends_on:
+      - keep-backend

docker/Dockerfile.dev.api

@@ -22,6 +22,7 @@ ENV PYTHONPATH="/app:${PYTHONPATH}"
 ENV PATH="/venv/bin:${PATH}"
 ENV VIRTUAL_ENV="/venv"
 ENV POSTHOG_DISABLED="true"
+ENV FRIGADE_DISABLED="true"
 
 ENTRYPOINT ["/app/keep/entrypoint.sh"]
 

docs/deployment/configuration.mdx

@@ -157,6 +157,18 @@ Sentry configuration controls Keep's integration with Sentry for error monitorin
 |:-------------------:|:-------:|:----------:|:-------------:|:-------------:|
 | **SENTRY_DISABLED** | Disables Sentry integration | No | "false" | "true" or "false" |
 
+
+### Frigade
+
+<Info>
+Frigade configuration controls Keep's integration with the Frigade onboarding platform.
+</Info>
+
+| Env var | Purpose | Required | Default Value | Valid options |
+|:-------------------:|:-------:|:----------:|:-------------:|:-------------:|
+| **FRIGADE_DISABLED** | Disables Frigade integration | No | "false" | "true" or "false" |
+
+
 ### Ngrok
 <Info>
 Ngrok configuration enables secure tunneling to your Keep instance. These settings are particularly useful for development or when you need to expose your local Keep instance to the internet securely.
@@ -208,6 +220,24 @@ ARQ (Asynchronous Task Queue) configuration controls Keep's background task proc
 | **ARQ_EXPIRES** | Default job expiration time (in seconds) | No | 3600 | Positive integer |
 | **ARQ_EXPIRES_AI** | AI job expiration time (in seconds) | No | 3600000 | Positive integer |
 
+### Rate Limiting
+<Info>
+Rate limiting configuration controls how many requests can be made to Keep's API endpoints within a specified time period. This helps prevent abuse and ensures system stability.
+</Info>
+
+| Env var | Purpose | Required | Default Value | Valid options |
+|:-------------------:|:-------:|:----------:|:-------------:|:-------------:|
+| **KEEP_USE_LIMITER** | Enables or disables rate limiting | No | "false" | "true" or "false" |
+| **KEEP_LIMIT_CONCURRENCY** | Sets the rate limit for API endpoints | No | "100/minute" | Format: "{number}/{interval}" where interval can be "second", "minute", "hour", "day" |
+
+<Note>
+Currently, rate limiting is applied to the following endpoints:
+- POST `/alerts/event` - Generic event ingestion endpoint
+- POST `/alerts/{provider_type}` - Provider-specific event ingestion endpoints
+
+These endpoints are rate-limited according to the `KEEP_LIMIT_CONCURRENCY` setting when `KEEP_USE_LIMITER` is enabled.
+</Note>
+
 ## Frontend Environment Variables
 <Info>
 Frontend configuration variables control the behavior and features of Keep's user interface. These settings are crucial for customizing the frontend's appearance, functionality, and integration with the backend services.

docs/deployment/kubernetes/overview.mdx

@@ -14,6 +14,6 @@ We maintain an opinionated, batteries-included Helm chart, but you can customize
 ## Next steps
 - Install Keep on [Kubernetes](/deployment/kubernetes/installation).
 - Keep's [Helm Chart](https://github.com/keephq/helm-charts).
-- Keep with [Kubernetes Secret Manager](/deployment/secret-manager#kubernetes-secret-manager)
+- Keep with [Kubernetes Secret Manager](/deployment/secret-store#kubernetes-secret-manager)
 - Deep dive to Keep's kubernetes [Architecture](/deployment/kubernetes/architecture).
 - Install Keep on [OpenShift](/deployment/kubernetes/openshift).

docs/deployment/provision/overview.mdx

@@ -8,10 +8,11 @@ Keep supports various deployment and provisioning strategies to accommodate diff
 
 Keep offers three main provisioning options:
 
-1. [**Provider Provisioning**](/deployment/provision/provider) - Set up and manage data providers for Keep.
+1. [**Provider Provisioning**](/deployment/provision/provider) - Set up and manage data providers with their deduplication rules for Keep.
 2. [**Workflow Provisioning**](/deployment/provision/workflow) - Configure and manage workflows within Keep.
 3. [**Dashboard Provisioning**](/deployment/provision/dashboard) - Configure and manage dashboards within Keep.
 
+
 Choosing the right provisioning strategy depends on your specific use case, deployment environment, and scalability requirements. You can read more about each provisioning option in their respective sections.
 
 ### How To Configure Provisioning
@@ -23,10 +24,11 @@ Choosing the right provisioning strategy depends on your specific use case, depl
 
 Provisioning in Keep is controlled through environment variables and configuration files. The main environment variables for provisioning are:
 
-| Provisioning Type | Environment Variable       | Purpose                                                |
-| ----------------- | -------------------------- | ------------------------------------------------------ |
-| **Provider**      | `KEEP_PROVIDERS`           | JSON string containing provider configurations         |
-| **Workflow**      | `KEEP_WORKFLOWS_DIRECTORY` | Directory path containing workflow configuration files |
-| **Dashboard**     | `KEEP_DASHBOARDS`          | JSON string containing dashboard configurations        |
+| Provisioning Type      | Environment Variable           | Purpose                                                                 |
+| ---------------------- | ------------------------------ | ----------------------------------------------------------------------- |
+| **Provider**           | `KEEP_PROVIDERS`               | JSON string containing provider configurations with deduplication rules |
+| **Workflow**           | `KEEP_WORKFLOWS_DIRECTORY`     | Directory path containing workflow configuration files                  |
+| **Dashboard**          | `KEEP_DASHBOARDS`              | JSON string containing dashboard configurations                         |
+
 
 For more details on each provisioning strategy, including setup instructions and implications, refer to the respective sections.

docs/deployment/provision/provider.mdx

@@ -8,15 +8,33 @@ Provider provisioning in Keep allows you to set up and manage data providers dyn
 
 ### Configuring Providers
 
-To provision providers, set the `KEEP_PROVIDERS` environment variable with a JSON string containing the provider configurations. Here's an example:
+To provision providers and deduplication rules for them, set the `KEEP_PROVIDERS` environment variable. This can be done in two ways:
+1. Directly with a JSON string containing the providers configurations.
+2. With the path to a JSON file that contains the providers configurations.
 
+<Tip>Please note: Deduplication rules are not mandatory for provider distribution. See the Clickhouse example.</Tip>
+
+Providers provisioning JSON example:
 ```json
 {
   "keepVictoriaMetrics": {
     "type": "victoriametrics",
     "authentication": {
       "VMAlertHost": "http://localhost",
-      "VMAlertPort": 1234
+      "VMAlertPort": 1234,
+      "deduplication_rules": {
+        "deduplication rule name example 1": {
+          "description": "deduplication rule name example 1",
+          "fingerprint_fields": ["fingerprint", "source", "service"],
+          "full_deduplication": true,
+          "ignore_fields": ["name", "lastReceived"]
+        },
+        "deduplication rule name example 2": {
+          "description": "deduplication rule name example 2",
+          "fingerprint_fields": ["fingerprint", "source", "service"],
+          "full_deduplication": false,
+        }
+      }
     }
   },
   "keepClickhouse1": {

docs/incidents/overview.mdx

@@ -21,10 +21,6 @@ A brief overview of the incident, optionally enhanced with AI-generated summarie
 ### (4) Link Similar Incidents
 Connects related incidents for better visibility into recurring or interconnected issues.
 
-<Frame width="100" height="200">
-  <img height="10" src="/images/incident_link.png" />
-</Frame>
-
 ### (5) Involved Services
 Lists the services affected by the incident, allowing teams to understand the scope of the impact.
 

docs/mint.json

@@ -210,6 +210,7 @@
     {
       "group": "Deployment",
       "pages": [
+        "deployment/getting-started",
         "deployment/configuration",
         "deployment/monitoring",
         {