-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle unexpected node reboots #421
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Sai Ramesh Vanka <[email protected]>
Skipping CI for Draft Pull Request. |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: sairameshv The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@sairameshv not sure if I got this correctly, but reading this comment #336 (comment) I get the impression that if we inject the code in deamonset to delete all allocations during the deamonset bootstrap, would it fix the issue? |
by that I meant, what happens if you fetch the corresponding instaslice object and delete all allocations somewhere here, https://github.com/openshift/instaslice-operator/blob/main/internal/controller/daemonset/instaslice_daemonset.go#L98 ? |
If we delete all the previous allocations when a daemonset comes up after a reboot, can we gate the pods again and create the slices ? Any thoughts @asm582 ? |
good point, all partitions will be deleted by the hardware after reboot. I think if an operator or deployment is used to spawn pods, then we will always have new ungated pods. we need to come up with a solution for plain pods. |
Potentially
Fixes #336
make test
,make lint
, pass locallymake test-e2e-kind-emulated
passes with some tweaks to the config files./hold