Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add slack alert when stopping EC2 instances fails in CI #725

Merged
merged 3 commits into from
Jun 13, 2024

Conversation

RomanBredehoft
Copy link
Collaborator

The last weekly CI failed for an unknown reason (internal error from GitHub), letting some EC2 instances running (which required to be manually terminated): https://github.com/zama-ai/concrete-ml/actions/runs/9468836256

this PR adds a slack alerts that will better warn us about such an issue cc @fd0r

@RomanBredehoft RomanBredehoft requested a review from a team as a code owner June 12, 2024 09:54
@cla-bot cla-bot bot added the cla-signed label Jun 12, 2024
@RomanBredehoft RomanBredehoft marked this pull request as draft June 12, 2024 09:57
@RomanBredehoft RomanBredehoft force-pushed the chore/add_slack_report_stop_ec2_failure branch from 81a65c8 to 1f33d46 Compare June 13, 2024 08:59
@RomanBredehoft RomanBredehoft force-pushed the chore/add_slack_report_stop_ec2_failure branch from 269d23d to cc32d3f Compare June 13, 2024 11:46
@RomanBredehoft RomanBredehoft marked this pull request as ready for review June 13, 2024 11:46
Copy link
Collaborator

@fd0r fd0r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Should we maybe add a ping to machinelearning to make sure that we don't miss it? Or even send it to the team slack channel and not the repo channel?

@RomanBredehoft
Copy link
Collaborator Author

RomanBredehoft commented Jun 13, 2024

@fd0r I tried to tag machinelearning but was not able to : https://zama-ai.slack.com/archives/C01022E8SRJ/p1718195645050339

currently, it tags the slack whole channel where these notifications are sent, like here : https://zama-ai.slack.com/archives/C02RX6CF3FV/p1718269669499519

are you saying you would prefer to have this notification in the main channel ? in that case I think we need another webhook and maybe other things, so might take a few more extra steps to do so

@RomanBredehoft RomanBredehoft merged commit 847d31c into main Jun 13, 2024
13 checks passed
@RomanBredehoft RomanBredehoft deleted the chore/add_slack_report_stop_ec2_failure branch June 13, 2024 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants