feat: servicenow cmdb and topology mapping

docs/overview/enrichment/mapping.mdx

@@ -4,28 +4,40 @@ title: "Mapping"
 
 # Alert Enrichment: Mapping
 
-Keep's Alert Mapping enrichment feature provides a powerful mechanism for dynamically enhancing alert data by leveraging external data sources, such as CSV files. This feature allows for the matching of incoming alerts to specific records in a CSV file based on predefined attributes (matchers) and enriching those alerts with additional information from the matched records.
+Keep's Alert Mapping enrichment feature provides a powerful mechanism for dynamically enhancing alert data by leveraging external data sources, such as CSV files and topology data. This feature allows for the matching of incoming alerts to specific records in a CSV file or topology data based on predefined attributes (matchers) and enriching those alerts with additional information from the matched records.
 
 ## Introduction
 
-In complex monitoring environments, the need to enrich alert data with additional context is critical for effective alert analysis and response. Keep's Alert Mapping and Enrichment enables users to define rules that match alerts to rows in a CSV file, appending or modifying alert attributes with the values from matching rows. This process adds significant value to each alert, providing deeper insights and enabling more precise and informed decision-making.
+In complex monitoring environments, the need to enrich alert data with additional context is critical for effective alert analysis and response. Keep's Alert Mapping and Enrichment enables users to define rules that match alerts to rows in a CSV file or topology data, appending or modifying alert attributes with the values from matching rows. This process adds significant value to each alert, providing deeper insights and enabling more precise and informed decision-making.
 
 ## How It Works
 
+## Mapping with CSV Files
+
 1. **Rule Definition**: Users define mapping rules that specify which alert attributes (matchers) should be used for matching alerts to rows in a CSV file.
 2. **CSV File Specification**: A CSV file is associated with each mapping rule. This file contains additional data that should be added to alerts matching the rule.
 3. **Alert Matching**: When an alert is received, the system checks if it matches the conditions of any mapping rule based on the specified matchers.
 4. **Data Enrichment**: If a match is found, the alert is enriched with additional data from the corresponding row in the CSV file.
 
+## Mapping with Topology Data
+
+1. **Rule Definition**: Users define mapping rules that specify which alert attributes (matchers) should be used for matching alerts to topology data.
+2. **Topology Data Specification**: Topology data is associated with each mapping rule. This data contains additional information about the components and their relationships in your environment.
+3. **Alert Matching**: When an alert is received, the system checks if it matches the conditions of any mapping rule based on the specified matchers.
+4. **Data Enrichment**: If a match is found, the alert is enriched with additional data from the corresponding topology data.
+
 ## Practical Example
 
 Imagine you have a CSV file with columns representing different aspects of your infrastructure, such as `region`, `responsible_team`, and `severity_override`. By creating a mapping rule that matches alerts based on `service` and `region`, you can automatically enrich alerts with the responsible team and adjust severity based on the matched row in the CSV file.
 
+Similarly, you can use topology data to enrich alerts. For example, if an alert is related to a specific service, you can use topology data to find related components and their statuses, providing a more comprehensive view of the issue.
+
 ## Core Concepts
 
-- **Matchers**: Attributes within the alert used to identify matching rows within the CSV file. Common matchers include identifiers like `service` or `region`.
+- **Matchers**: Attributes within the alert used to identify matching rows within the CSV file or topology data. Common matchers include identifiers like `service` or `region`.
 - **CSV File**: A structured file containing rows of data. Each column represents a potential attribute that can be added to an alert.
-- **Enrichment**: The process of adding new attributes or modifying existing ones in an alert based on the data from a matching CSV row.
+- **Topology Data**: Information about the components and their relationships in your environment. This data can be used to enrich alerts with additional context.
+- **Enrichment**: The process of adding new attributes or modifying existing ones in an alert based on the data from a matching CSV row or topology data.
 
 ## Creating a Mapping Rule
 
@@ -35,13 +47,13 @@ To create an alert mapping and enrichment rule:
   <img height="10" src="/images/rule-creation.png" />
 </Frame>
 
-1. **Define the Matchers**: Specify which alert attributes will be used to match rows in the CSV file.
-2. **Upload the CSV File**: Provide the CSV file containing the data for enrichment.
+1. **Define the Matchers**: Specify which alert attributes will be used to match rows in the CSV file or topology data.
+2. **Specify the Data Source**: Provide the CSV file or specify the topology data to be used for enrichment.
 3. **Configure the Rule**: Set additional parameters, such as whether the rule should override existing alert attributes.
 
 ## Best Practices
 
-- **Keep CSV Files Updated**: Regularly update the CSV files to reflect the current state of your infrastructure and operational data.
+- **Keep CSV Files and Topology Data Updated**: Regularly update the CSV files and topology data to reflect the current state of your infrastructure and operational data.
 - **Use Specific Matchers**: Define matchers that are unique and relevant to ensure accurate matching.
 - **Monitor Rule Performance**: Review the application of mapping rules to ensure they are working as expected and adjust them as necessary.
 

keep-ui/app/alerts/alert-sidebar.tsx

@@ -112,7 +112,6 @@ const AlertSidebar = ({ isOpen, toggle, alert }: AlertSidebarProps) => {
                   providerId={alert.providerId || ""}
                   service={alert.service || ""}
                   environment={"unknown"}
-                  showSearch={false}
                 />
               </div>
             )}

keep-ui/app/mapping/create-or-edit-mapping.tsx

@@ -11,6 +11,11 @@ import {
   Badge,
   Button,
   Icon,
+  TabGroup,
+  TabList,
+  Tab,
+  TabPanels,
+  TabPanel,
 } from "@tremor/react";
 import { useSession } from "next-auth/react";
 import {
@@ -27,6 +32,7 @@ import { getApiURL } from "utils/apiUrl";
 import { useMappings } from "utils/hooks/useMappingRules";
 import { MappingRule } from "./models";
 import { CreateableSearchSelect } from "@/components/ui/CreateableSearchSelect";
+import { useTopology } from "utils/hooks/useTopology";
 
 interface Props {
   editRule: MappingRule | null;
@@ -36,12 +42,15 @@ interface Props {
 export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
   const { data: session } = useSession();
   const { mutate } = useMappings();
+  const [tabIndex, setTabIndex] = useState<number>(0);
   const [mapName, setMapName] = useState<string>("");
   const [fileName, setFileName] = useState<string>("");
+  const [mappingType, setMappingType] = useState<"csv" | "topology">("csv");
   const [mapDescription, setMapDescription] = useState<string>("");
   const [selectedLookupAttributes, setSelectedLookupAttributes] = useState<
     string[]
   >([]);
+  const { topologyData } = useTopology();
   const [priority, setPriority] = useState<number>(0);
   const editMode = editRule !== null;
   const inputFile = useRef<HTMLInputElement>(null);
@@ -53,6 +62,8 @@ export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
       setMapName(editRule.name);
       setFileName(editRule.file_name ? editRule.file_name : "");
       setMapDescription(editRule.description ? editRule.description : "");
+      setMappingType(editRule.type ? editRule.type : "csv");
+      setTabIndex(editRule.type === "csv" ? 0 : 1);
       setSelectedLookupAttributes(editRule.matchers ? editRule.matchers : []);
       setPriority(editRule.priority);
     }
@@ -83,6 +94,17 @@ export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
     }
   };
 
+  const updateMappingType = (index: number) => {
+    setTabIndex(index);
+    if (index === 0) {
+      setParsedData(null);
+      setMappingType("csv");
+    } else {
+      setParsedData(topologyData!);
+      setMappingType("topology");
+    }
+  };
+
   const readFile = (event: ChangeEvent<HTMLInputElement>) => {
     const file = event.target.files?.[0];
     setFileName(file?.name || "");
@@ -123,8 +145,9 @@ export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
         name: mapName,
         description: mapDescription,
         file_name: fileName,
+        type: mappingType,
         matchers: selectedLookupAttributes.map((attr) => attr.trim()),
-        rows: parsedData,
+        rows: mappingType === "csv" ? parsedData : null,
       }),
     });
     if (response.ok) {
@@ -142,7 +165,7 @@ export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
   const updateRule = async (e: FormEvent) => {
     e.preventDefault();
     const apiUrl = getApiURL();
-    const response = await fetch(`${apiUrl}/mapping`, {
+    const response = await fetch(`${apiUrl}/mapping/${editRule?.id}`, {
       method: "PUT",
       headers: {
         Authorization: `Bearer ${session?.accessToken}`,
@@ -154,8 +177,9 @@ export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
         name: mapName,
         description: mapDescription,
         file_name: fileName,
+        type: mappingType,
         matchers: selectedLookupAttributes.map((attr) => attr.trim()),
-        rows: parsedData,
+        rows: mappingType === "csv" ? parsedData : null,
       }),
     });
     if (response.ok) {
@@ -229,20 +253,45 @@ export default function CreateOrEditMapping({ editRule, editCallback }: Props) {
       </div>
       <Divider />
       <div>
-        <input
-          type="file"
-          accept=".csv, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, application/vnd.ms-excel"
-          onChange={readFile}
-          required={!editMode}
-          ref={inputFile}
-        />
-        {!parsedData && (
-          <Text className="text-xs text-red-500">
-            {!editMode
-              ? "* Upload a CSV file to begin with creating a new mapping"
-              : ""}
-          </Text>
-        )}
+        <TabGroup
+          index={tabIndex}
+          onIndexChange={(index) => updateMappingType(index)}
+        >
+          <TabList>
+            <Tab>CSV</Tab>
+            <Tab
+              disabled={!topologyData || topologyData.length === 0}
+              className={`${
+                !topologyData || topologyData.length === 0
+                  ? "text-gray-400"
+                  : ""
+              }`}
+            >
+              Topology
+            </Tab>
+          </TabList>
+          <TabPanels>
+            <TabPanel>
+              {mappingType === "csv" && (
+                <input
+                  type="file"
+                  accept=".csv, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, application/vnd.ms-excel"
+                  onChange={readFile}
+                  required={!editMode}
+                  ref={inputFile}
+                />
+              )}
+              {!parsedData && (
+                <Text className="text-xs text-red-500">
+                  {!editMode
+                    ? "* Upload a CSV file to begin with creating a new mapping"
+                    : ""}
+                </Text>
+              )}
+            </TabPanel>
+            <TabPanel></TabPanel>
+          </TabPanels>
+        </TabGroup>
       </div>
       <Subtitle className="mt-2.5">Mapping Schema</Subtitle>
       <div className="mt-2.5">

keep-ui/app/mapping/mapping.tsx

@@ -36,10 +36,8 @@ export default function Mapping() {
               title="Mapping rules does not exist"
               icon={MdWarning}
             >
-              <p className="text-slate-400">No mapping rules found</p>
-              <p className="text-slate-400">
-                Configure new mapping rule using the mapping rules wizard.
-              </p>
+              No mapping rules found. Configure new mapping rule using the
+              mapping rules wizard.
             </Callout>
           )}
         </div>

keep-ui/app/mapping/models.tsx

@@ -11,6 +11,7 @@ export interface MappingRule {
   last_updated_at: Date;
   disabled: boolean;
   override: boolean;
+  type: "csv" | "topology";
   condition?: string;
   matchers: string[];
   rows: { [key: string]: any }[];

keep-ui/app/mapping/rules-table.tsx

@@ -70,6 +70,11 @@ export default function RulesTable({ mappings, editCallback }: Props) {
       header: "Name",
       cell: (context) => context.row.original.name,
     }),
+    columnHelper.display({
+      id: "type",
+      header: "Type",
+      cell: (context) => context.row.original.type,
+    }),
     columnHelper.display({
       id: "description",
       header: "Description",

keep-ui/app/topology/custom-node.tsx

@@ -1,4 +1,4 @@
-import React, { useEffect } from "react";
+import React, { useEffect, useState } from "react";
 import { Handle, Position } from "@xyflow/react";
 import { useAlerts } from "utils/hooks/useAlerts";
 import { useAlertPolling } from "utils/hooks/usePusher";
@@ -12,14 +12,17 @@ const CustomNode = ({ data }: { data: TopologyService }) => {
   const { data: alerts, mutate } = useAllAlerts("feed");
   const { data: pollAlerts } = useAlertPolling();
   const router = useRouter();
+  const [showDetails, setShowDetails] = useState(false);
 
   useEffect(() => {
     if (pollAlerts) {
       mutate();
     }
   }, [pollAlerts, mutate]);
 
-  const relevantAlerts = alerts?.filter((alert) => alert.service === data.service);
+  const relevantAlerts = alerts?.filter(
+    (alert) => alert.service === data.service
+  );
 
   const handleClick = () => {
     router.push(
@@ -31,8 +34,12 @@ const CustomNode = ({ data }: { data: TopologyService }) => {
   const badgeColor = alertCount < THRESHOLD ? "bg-orange-500" : "bg-red-500";
 
   return (
-    <div className="bg-white p-4 border rounded-xl shadow-lg relative">
-      <strong className="text-lg">{data.service}</strong>
+    <div
+      className="bg-white p-4 border rounded-xl shadow-lg relative"
+      onMouseEnter={() => setShowDetails(true)}
+      onMouseLeave={() => setShowDetails(false)}
+    >
+      <strong className="text-lg">{data.display_name ?? data.service}</strong>
       {alertCount > 0 && (
         <span
           className={`absolute top-[-20px] right-[-20px] mt-2 mr-2 px-2 py-1 text-white text-xs font-bold rounded-full ${badgeColor} hover:cursor-pointer`}
@@ -41,6 +48,60 @@ const CustomNode = ({ data }: { data: TopologyService }) => {
           {alertCount}
         </span>
       )}
+      {showDetails && (
+        <div className="absolute top-full left-1/2 transform -translate-x-1/2 mt-2 p-4 bg-white border rounded-xl shadow-lg z-50">
+          {data.service && (
+            <p>
+              <strong>Service:</strong> {data.service}
+            </p>
+          )}
+          {data.display_name && (
+            <p>
+              <strong>Display Name:</strong> {data.display_name}
+            </p>
+          )}
+          {data.description && (
+            <p>
+              <strong>Description:</strong> {data.description}
+            </p>
+          )}
+          {data.team && (
+            <p>
+              <strong>Team:</strong> {data.team}
+            </p>
+          )}
+          {data.application && (
+            <p>
+              <strong>Application:</strong> {data.application}
+            </p>
+          )}
+          {data.email && (
+            <p>
+              <strong>Email:</strong> {data.email}
+            </p>
+          )}
+          {data.slack && (
+            <p>
+              <strong>Slack:</strong> {data.slack}
+            </p>
+          )}
+          {data.ip_address && (
+            <p>
+              <strong>IP Address:</strong> {data.ip_address}
+            </p>
+          )}
+          {data.mac_address && (
+            <p>
+              <strong>MAC Address:</strong> {data.mac_address}
+            </p>
+          )}
+          {data.manufacturer && (
+            <p>
+              <strong>Manufacturer:</strong> {data.manufacturer}
+            </p>
+          )}
+        </div>
+      )}
       <Handle type="source" position={Position.Right} id="right" />
       <Handle type="target" position={Position.Left} id="left" />
     </div>

keep-ui/app/topology/layout.tsx

@@ -1,5 +1,30 @@
+"use client";
+import { Subtitle, TextInput, Title } from "@tremor/react";
+import { useState } from "react";
+import { ServiceSearchContext } from "./service-search-context";
+
 export default function Layout({ children }: { children: any }) {
+  const [serviceInput, setServiceInput] = useState<string>("");
+
   return (
-    <main className="p-4 md:p-10 mx-auto max-w-full h-full">{children}</main>
+    <main className="p-4 md:p-10 mx-auto max-w-full h-full">
+      <div className="flex w-full justify-between">
+        <div>
+          <Title>Service Topology</Title>
+          <Subtitle>
+            Data describing the topology of components in your environment.
+          </Subtitle>
+        </div>
+        <TextInput
+          placeholder="Search for a service"
+          value={serviceInput}
+          onValueChange={setServiceInput}
+          className="w-64 mt-2"
+        />
+      </div>
+      <ServiceSearchContext.Provider value={serviceInput}>
+        {children}
+      </ServiceSearchContext.Provider>
+    </main>
   );
 }

keep-ui/app/topology/models.tsx

@@ -17,4 +17,7 @@ export interface TopologyService {
   email?: string;
   slack?: string;
   dependencies: TopologyServiceDependency[];
+  ip_address?: string;
+  mac_address?: string;
+  manufacturer?: string;
 }