Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e: fix runtime reproducibility test and also check kata.contrast-node-installer-image #960

Merged
merged 2 commits into from
Oct 31, 2024

Conversation

miampf
Copy link
Contributor

@miampf miampf commented Oct 29, 2024

Follow up of #947. New successful run can be found here.

@miampf miampf requested a review from Freax13 October 29, 2024 11:48
@miampf miampf requested a review from katexochen as a code owner October 29, 2024 11:48
@miampf miampf added the no changelog PRs not listed in the release notes label Oct 29, 2024
.github/workflows/e2e_runtime-reproducibility.yml Outdated Show resolved Hide resolved
.github/workflows/e2e_runtime-reproducibility.yml Outdated Show resolved Hide resolved
.github/workflows/e2e_runtime-reproducibility.yml Outdated Show resolved Hide resolved
.github/workflows/e2e_runtime-reproducibility.yml Outdated Show resolved Hide resolved
@miampf miampf force-pushed the miampf/e2e/reproducible-kata branch from 60107d4 to f5d6e63 Compare October 29, 2024 14:29
@miampf miampf force-pushed the miampf/e2e/reproducible-kata branch from fc0bf1c to a41cce6 Compare October 29, 2024 15:23
@katexochen
Copy link
Member

@miampf could you separate the fix and the additional test into two distinct commits please (and rebase-merge later)?

@miampf
Copy link
Contributor Author

miampf commented Oct 31, 2024

@miampf could you separate the fix and the additional test into two distinct commits please (and rebase-merge later)?

It will take some time until merge anyways since the test is not failing as expected. When it comes to merging it I will keep this in mind tho :)

@miampf miampf force-pushed the miampf/e2e/reproducible-kata branch from 70a2682 to a41cce6 Compare October 31, 2024 11:55
@miampf
Copy link
Contributor Author

miampf commented Oct 31, 2024

The test does fail if different values are detected (look here, I just randomly generated strings instead of using a hash), however, it does not fail when comparing the images digests (look here), reason being that they are equal. @katexochen, since reproducibility should be broken but the digests are the same, should we compare something other than the image digest? If yes, what do you think would be a good option?

@miampf miampf force-pushed the miampf/e2e/reproducible-kata branch from a41cce6 to 36de831 Compare October 31, 2024 12:04
@katexochen
Copy link
Member

katexochen, since reproducibility should be broken but the digests are the same, should we compare something other than the image digest? If yes, what do you think would be a good option?

It's fine, the current reproducibility issues are really hard to reproduce, it would be to much work to detect that in a ci job. The test succeeds as this reproduces between the two machine, so there is no blocker to merge this from my perspective.

@miampf miampf force-pushed the miampf/e2e/reproducible-kata branch from 36de831 to 21f5a5a Compare October 31, 2024 12:28
@miampf miampf force-pushed the miampf/e2e/reproducible-kata branch from 21f5a5a to d0a469c Compare October 31, 2024 12:38
@miampf miampf merged commit 74f5abe into main Oct 31, 2024
9 checks passed
@miampf miampf deleted the miampf/e2e/reproducible-kata branch October 31, 2024 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no changelog PRs not listed in the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants