Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add slack alert when stopping EC2 instances fails in CI #725

Merged
merged 3 commits into from
Jun 13, 2024
Merged

chore: add slack alert when stopping EC2 instances fails in CI #725

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cf2574b
chore: add slack alert when stopping EC2 instances fails in CI
RomanBredehoft Jun 12, 2024
34c7059
chore: trigger error message
RomanBredehoft Jun 12, 2024
cc32d3f
chore: revert trigger error step
RomanBredehoft Jun 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions .github/workflows/continuous-integration.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -718,6 +718,7 @@ jobs:
)
&& steps.conformance.outcome == 'success'
&& !cancelled()

shell: bash +e {0}
run: |
if [[ "${{ env.IS_WEEKLY }}" == "true" ]]; then
Expand Down Expand Up @@ -1112,8 +1113,8 @@ jobs:
# basically when the `matrix-preparation` has not been skipped
# Side note: environmental variables cannot be used for jobs conditions, so we need to determine
# if the job should be run or not in an previous job and store it in its output
send-report:
name: Send Slack report
slack-report:
name: Slack report
runs-on: ubuntu-20.04
if: |
always()
Expand Down Expand Up @@ -1198,7 +1199,7 @@ jobs:

echo "EOF" >> "$GITHUB_ENV"

- name: Slack report
- name: Send Slack report
uses: rtCamp/action-slack-notify@4e5fb42d249be6a45a298f3c9543b111b02f7907
env:
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
Expand All @@ -1210,3 +1211,29 @@ jobs:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
SLACKIFY_MARKDOWN: true


stop-ec2-failure-slack-alert:
name: Stop EC2 failure Slack alert
runs-on: ubuntu-20.04
if: |
always()
&& needs.matrix-preparation.result != 'skipped'
&& needs.stop-runner-linux.result == 'failure'
timeout-minutes: 2
needs:
[
matrix-preparation,
stop-runner-linux,
]

steps:
- name: Send stop EC2 failure Slack alert
uses: rtCamp/action-slack-notify@4e5fb42d249be6a45a298f3c9543b111b02f7907
env:
SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}
SLACK_ICON: https://pbs.twimg.com/profile_images/1274014582265298945/OjBKP9kn_400x400.png
SLACK_COLOR: 'failure'
SLACK_TITLE: 'Stopping EC2 instances (linux) failed ❌'
SLACK_MESSAGE: '<!channel> EC2 instances must be terminated manually as soon as possible'
SLACK_USERNAME: ${{ secrets.BOT_USERNAME }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
Loading