Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: a couple of issues with extraction #2250

Merged
merged 2 commits into from
Oct 20, 2024
Merged

fix: a couple of issues with extraction #2250

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dce59ba
fix: a couple of issues with extraction
talboren Oct 20, 2024
b3a9743
fix: workflow for developer onboarding
talboren Oct 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 16 additions & 25 deletions ...ows/developer_onboarding_notification.yml → ...ows/developer-onboarding-notification.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ jobs:
try {
const prCount = parseInt(process.env.PR_COUNT);
const author = process.env.AUTHOR;
const mention = 'talboren';
const prNumber = context.payload.pull_request.number;
const repo = process.env.REPO;

Expand All @@ -68,49 +67,41 @@ jobs:
return `${emoji} **Fantastic work @${author}!** Your very first PR to ${repo} has been merged! 🎉🥳\n\n` +
`You've just taken your first step into open-source, and we couldn't be happier to have you onboard. 🙌\n` +
`If you're feeling adventurous, why not dive into another issue and keep contributing? The community would love to see more from you! 🚀\n\n` +
`For any support, feel free to reach out to the developer onboarding lead: @${mention}. Happy coding! 👩‍💻👨‍💻`;
`For any support, feel free to reach out on the community: https://slack.keephq.dev. Happy coding! 👩‍💻👨‍💻`;
case 2:
return `${emoji} **Well done @${author}!** Two PRs merged already! 🎉🥳\n\n` +
`With your second PR, you're on a roll, and your contributions are already making a difference. 🌟\n` +
`Looking forward to seeing even more contributions from you. The developer onboarding lead: @${mention} is here if you need any help! Keep up the great work! 🚀`;
`Looking forward to seeing even more contributions from you. See you in Slack https://slack.keephq.dev 🚀`;
case 3:
return `${emoji} **You're on fire, @${author}!** Three PRs merged and counting! 🔥🎉\n\n` +
`Your consistent contributions are truly impressive. You're becoming a valued member of our community! 💖\n` +
`Have you considered taking on some more challenging issues? We'd love to see what you can do! 💪\n\n` +
`Remember, @${mention} is always here to support you. Keep blazing that trail! 🚀`;
`Remember, the team is always here to support you. Keep blazing that trail! 🚀`;
case 5:
return `${emoji} **High five, @${author}!** You've hit the incredible milestone of 5 merged PRs! 🖐️✨\n\n` +
`Your dedication to ${repo} is outstanding. You're not just contributing code; you're shaping the future of this project! 🌠\n` +
`We'd love to hear your thoughts on the project. Any ideas for new features or improvements? 🤔\n\n` +
`@${mention} and the whole team applaud your efforts. You're a superstar! 🌟`;
`The whole team applaud your efforts. You're a superstar! 🌟`;
case 10:
return `${emoji} **Double digits, @${author}!** 10 merged PRs is a massive achievement! 🏆🎊\n\n` +
`Your impact on ${repo} is undeniable. You've become a pillar of our community! 🏛️\n` +
`We'd be thrilled to have you take on a mentorship role for newer contributors. Interested? 🧑‍🏫\n\n` +
`@${mention} and everyone here are in awe of your contributions. You're an open source hero! 🦸‍♀️🦸‍♂️`;
`Everyone here are in awe of your contributions. You're an open source hero! 🦸‍♀️🦸‍♂️`;
default:
if (count > 10) {
return `${emoji} **Incredible, @${author}!** You've merged your ${count}th PR! 🎯🎊\n\n` +
`Your ongoing commitment to ${repo} is truly remarkable. You're a driving force in our community! 🚀\n` +
`Your contributions are helping to shape the future of this project. What exciting features or improvements do you envision next? 🔮\n\n` +
`@${mention} and the entire team are grateful for your dedication. You're an inspiration to us all! 💫`;
} else {
return `${emoji} **Great job, @${author}!** You've merged your ${count}th PR! 🎊\n\n` +
`Your contributions to ${repo} are making a real difference. Keep up the fantastic work! 💪\n` +
`Remember, every PR counts and helps improve the project. What will you tackle next? 🤔\n\n` +
`@${mention} is here if you need any guidance. Onward and upward! 🚀`;
}
return "";
}
}

const message = getMessage(prCount);

await github.rest.issues.createComment({
owner: process.env.OWNER,
repo: process.env.REPO,
issue_number: prNumber,
body: message
});

if (message) {
await github.rest.issues.createComment({
owner: process.env.OWNER,
repo: process.env.REPO,
issue_number: prNumber,
body: message
});
}
} catch (error) {
core.setFailed(`Error creating comment: ${error.message}`);
}
}
156 changes: 88 additions & 68 deletions keep-ui/app/alerts/alerts-rules-builder.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { useEffect, useRef, useState } from "react";
import { useCallback, useEffect, useRef, useState } from "react";
import Modal from "components/ui/Modal";
import { Button, Textarea, Badge } from "@tremor/react";
import QueryBuilder, {
Expand All @@ -12,7 +12,12 @@ import QueryBuilder, {
} from "react-querybuilder";
import "react-querybuilder/dist/query-builder.scss";
import { Table } from "@tanstack/react-table";
import { AlertDto, Preset, severityMapping, reverseSeverityMapping } from "./models";
import {
AlertDto,
Preset,
severityMapping,
reverseSeverityMapping,
} from "./models";
import { XMarkIcon, TrashIcon } from "@heroicons/react/24/outline";
import { FiSave } from "react-icons/fi";
import { TbDatabaseImport } from "react-icons/tb";
Expand All @@ -21,6 +26,7 @@ import Select, { components, MenuListProps } from "react-select";
import { IoSearchOutline } from "react-icons/io5";
import { FiExternalLink } from "react-icons/fi";
import { usePathname, useRouter, useSearchParams } from "next/navigation";
import { toast } from "react-toastify";

const staticOptions = [
{ value: 'severity > "info"', label: 'severity > "info"' },
Expand Down Expand Up @@ -127,68 +133,68 @@ const getAllMatches = (pattern: RegExp, string: string) =>
// make sure string is a String, and make sure pattern has the /g flag
String(string).match(new RegExp(pattern, "g"));

const sanitizeCELIntoJS = (celExpression: string): string => {
// First, replace "contains" with "includes"
let jsExpression = celExpression.replace(/contains/g, "includes");

// Replace severity comparisons with mapped values
jsExpression = jsExpression.replace(
/severity\s*([<>]=?|==)\s*(\d+|"[^"]*")/g,
(match, operator, value) => {
let severityKey;

if (/^\d+$/.test(value)) {
// If the value is a number
severityKey = severityMapping[Number(value)];
} else {
// If the value is a string
severityKey = value.replace(/"/g, '').toLowerCase(); // Remove quotes from the string value and convert to lowercase
}

const severityValue = reverseSeverityMapping[severityKey];

if (severityValue === undefined) {
return match; // If no mapping found, return the original match
}

// For equality, directly replace with the severity level
if (operator === "==") {
return `severity == "${severityKey}"`;
}

// For greater than or less than, include multiple levels based on the mapping
const levels = Object.entries(reverseSeverityMapping);
let replacement = "";
if (operator === ">") {
const filteredLevels = levels
.filter(([, level]) => level > severityValue)
.map(([key]) => `severity == "${key}"`);
replacement = filteredLevels.join(" || ");
} else if (operator === "<") {
const filteredLevels = levels
.filter(([, level]) => level < severityValue)
.map(([key]) => `severity == "${key}"`);
replacement = filteredLevels.join(" || ");
}

return `(${replacement})`;
const sanitizeCELIntoJS = (celExpression: string): string => {
// First, replace "contains" with "includes"
let jsExpression = celExpression.replace(/contains/g, "includes");

// Replace severity comparisons with mapped values
jsExpression = jsExpression.replace(
/severity\s*([<>]=?|==)\s*(\d+|"[^"]*")/g,
(match, operator, value) => {
let severityKey;

if (/^\d+$/.test(value)) {
// If the value is a number
severityKey = severityMapping[Number(value)];
} else {
// If the value is a string
severityKey = value.replace(/"/g, "").toLowerCase(); // Remove quotes from the string value and convert to lowercase
}
);

// Convert 'in' syntax to '.includes()'
jsExpression = jsExpression.replace(
/(\w+)\s+in\s+\[([^\]]+)\]/g,
(match, variable, list) => {
// Split the list by commas, trim spaces, and wrap items in quotes if not already done
const items = list
.split(",")
.map((item: string) => item.trim().replace(/^([^"]*)$/, '"$1"'));
return `[${items.join(", ")}].includes(${variable})`;
const severityValue = reverseSeverityMapping[severityKey];

if (severityValue === undefined) {
return match; // If no mapping found, return the original match
}
);

return jsExpression;
};
// For equality, directly replace with the severity level
if (operator === "==") {
return `severity == "${severityKey}"`;
}

// For greater than or less than, include multiple levels based on the mapping
const levels = Object.entries(reverseSeverityMapping);
let replacement = "";
if (operator === ">") {
const filteredLevels = levels
.filter(([, level]) => level > severityValue)
.map(([key]) => `severity == "${key}"`);
replacement = filteredLevels.join(" || ");
} else if (operator === "<") {
const filteredLevels = levels
.filter(([, level]) => level < severityValue)
.map(([key]) => `severity == "${key}"`);
replacement = filteredLevels.join(" || ");
}

return `(${replacement})`;
}
);

// Convert 'in' syntax to '.includes()'
jsExpression = jsExpression.replace(
/(\w+)\s+in\s+\[([^\]]+)\]/g,
(match, variable, list) => {
// Split the list by commas, trim spaces, and wrap items in quotes if not already done
const items = list
.split(",")
.map((item: string) => item.trim().replace(/^([^"]*)$/, '"$1"'));
return `[${items.join(", ")}].includes(${variable})`;
}
);

return jsExpression;
};

// this pattern is far from robust
const variablePattern = /[a-zA-Z$_][0-9a-zA-Z$_]*/;
Expand Down Expand Up @@ -278,6 +284,8 @@ type AlertsRulesBuilderProps = {
customFields?: Field[];
showSave?: boolean;
minimal?: boolean;
showToast?: boolean;
shouldSetQueryParam?: boolean;
};

const SQL_QUERY_PLACEHOLDER = `SELECT *
Expand All @@ -296,6 +304,8 @@ export const AlertsRulesBuilder = ({
showSqlImport = true,
showSave = true,
minimal = false,
showToast = false,
shouldSetQueryParam = true,
}: AlertsRulesBuilderProps) => {
const router = useRouter();
const pathname = usePathname();
Expand Down Expand Up @@ -336,6 +346,12 @@ export const AlertsRulesBuilder = ({

const [showSuggestions, setShowSuggestions] = useState(false);

const handleClearInput = useCallback(() => {
setCELRules("");
table?.resetGlobalFilter();
setIsValidCEL(true);
}, [table]);

const toggleSuggestions = () => {
setShowSuggestions(!showSuggestions);
};
Expand Down Expand Up @@ -385,6 +401,14 @@ export const AlertsRulesBuilder = ({
};
}, []);

useEffect(() => {
if (defaultQuery === "") {
handleClearInput();
} else {
setCELRules(defaultQuery);
}
}, [defaultQuery, handleClearInput]);

useEffect(() => {
// Use the constructCELRules function to set the initial value of celRules
const initialCELRules = constructCELRules(selectedPreset);
Expand Down Expand Up @@ -422,12 +446,6 @@ export const AlertsRulesBuilder = ({
adjustTextAreaHeight();
}, [celRules]);

const handleClearInput = () => {
setCELRules("");
table?.resetGlobalFilter();
setIsValidCEL(true);
};

const handleKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (e.key === "Enter") {
e.preventDefault(); // Prevents the default action of Enter key in a form
Expand All @@ -443,9 +461,11 @@ export const AlertsRulesBuilder = ({
// close the menu
setShowSuggestions(false);
if (isValidCEL) {
setQueryParam("cel", celRules);
if (shouldSetQueryParam) setQueryParam("cel", celRules);
onApplyFilter();
updateOutputCEL?.(celRules);
if (showToast)
toast.success("Condition applied", { position: "top-right" });
}
}
};
Expand Down Expand Up @@ -629,7 +649,7 @@ export const AlertsRulesBuilder = ({
options={staticOptions}
onChange={handleSelectChange}
menuIsOpen={true}
components={minimal? undefined: customComponents}
components={minimal ? undefined : customComponents}
onBlur={() => setShowSuggestions(false)}
styles={customStyles}
/>
Expand Down
7 changes: 4 additions & 3 deletions keep-ui/app/extraction/create-or-update-extraction-rule.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,15 +129,14 @@ export default function CreateOrUpdateExtractionRule({
if (response.ok) {
exitEditMode();
mutate();
toast.success("Mapping updated successfully");
toast.success("Extraction updated successfully");
} else {
toast.error(
"Failed to update mapping, please contact us if this issue persists."
"Failed to update extraction, please contact us if this issue persists."
);
}
};

// If the mapping is successfully updated or the user cancels the update we exit the editMode and set the editRule in the mapping.tsx to null.
const exitEditMode = async () => {
editCallback(null);
clearForm();
Expand Down Expand Up @@ -280,6 +279,8 @@ export default function CreateOrUpdateExtractionRule({
updateOutputCEL={setCondition}
showSave={false}
showSqlImport={false}
showToast={true}
shouldSetQueryParam={false}
/>
</div>
</div>
Expand Down
2 changes: 0 additions & 2 deletions keep/api/bl/enrichments_bl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,6 @@ def run_extraction_rules(
"fingerprint": fingerprint,
},
)
# Stop after the first match
break
else:
self.logger.info(
"Regex did not match, skipping extraction",
Expand Down
Loading