Skip to content

Commit

Permalink
GP-4477 inhibit on AutoInhibit alert for workflow run ID
Browse files Browse the repository at this point in the history
  • Loading branch information
Heather Armstrong authored and callunity committed Dec 4, 2024
1 parent dad531f commit f3962b5
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 9 deletions.
1 change: 1 addition & 0 deletions changes/add_inhibit_on_workflow_run_id_alert.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Inhibit workflow run launch if AutoInhibit alert contains workflow run ID in its job or scope
9 changes: 5 additions & 4 deletions vidarr-prometheus/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ Alertmanager times out.
workflow run inhibition.

`"valuesOfInterest"` is the set of label values that will be assessed for
workflow run inhibition. Internally, the _workflow name_ and _workflow name and version_*
are added to this list of values of interest.
workflow run inhibition. Internally, the _workflow name_ and _workflow name and version_*,
and _workflow run ID_ are added to this list of values of interest.

### Workflow run inhibition
In order for a match and a workflow run inhibition to occur, the alert must
Expand All @@ -63,10 +63,11 @@ have the following:
* one or more of the labels in `<labelsOfInterest>` whose value is:
* one of the `<valuesOfInterest>`; or
* _workflow name_; or
* _workflow name and version_*
* _workflow name and version_* ; or
* _workflow run ID_

*(formatted like
_bcl2fastq_3_2_0_, where any periods in the version are converted to underscores)

The workflow run inhibition will last as long as the AutoInhibit alert is
firing.
firing.
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ public ConsumableResourceResponse request(
Stream.of(
valuesOfInterest.stream(),
Stream.of(workflowName),
Stream.of(vidarrId),
Stream.of(
String.format(
"%s_%s", workflowName, workflowVersionWithUnderscores)))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public void whenEnvAndJobMatch_matchesShouldMatch() {

var matches =
sut.matches(autoInhibit, "testing", configLabels, Stream.of("vidarr-clinical", "bamqc4"))
.collect(Collectors.toList());
.toList();
assertEquals(1, matches.size());
}

Expand All @@ -44,7 +44,7 @@ public void whenEnvAndWorkflowMatch_matchesShouldMatch() {

var matches =
sut.matches(autoInhibit, "testing", configLabels, Stream.of("vidarr-clinical", "bamqc4"))
.collect(Collectors.toList());
.toList();
assertEquals(1, matches.size());
}

Expand All @@ -61,7 +61,7 @@ public void whenEnvAndScopeMatch_matchesShouldMatch() {

var matches =
sut.matches(autoInhibit, "testing", configLabels, Stream.of("vidarr-clinical", "bamqc4"))
.collect(Collectors.toList());
.toList();
assertEquals(1, matches.size());
}

Expand All @@ -78,7 +78,7 @@ public void whenEnvDoesNotMatch_matchesShouldNotMatch() {

var matches =
sut.matches(autoInhibit, "testing", configLabels, Stream.of("vidarr-clinical", "bamqc4"))
.collect(Collectors.toList());
.toList();
assertEquals(0, matches.size());
}

Expand All @@ -95,7 +95,24 @@ public void whenAlertnameDoesNotMatch_matchesShouldNotMatch() {

var matches =
sut.matches(autoInhibit, "testing", configLabels, Stream.of("vidarr-clinical", "bamqc4"))
.collect(Collectors.toList());
.toList();
assertEquals(0, matches.size());
}

@Test
public void whenAlertScopeIsWorkflowRunId_matchesShouldMatch() {
AlertDto sut = new AlertDto();

ObjectNode labels = mapper.createObjectNode();
labels.put("environment", "testing");
labels.put("job", "bamqc4");
labels.put("scope", "615ed228fad3ae6193d5279dc689e83fa4225cd69c929e266dd84ef2ed96e719");
labels.put("alertname", "AutoInhibit");
sut.setLabels(labels);

var matches =
sut.matches(autoInhibit, "testing", configLabels, Stream.of("vidarr-clinical", "615ed228fad3ae6193d5279dc689e83fa4225cd69c929e266dd84ef2ed96e719"))
.toList();
assertEquals(1, matches.size());
}
}

0 comments on commit f3962b5

Please sign in to comment.