-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Makefile: Minor cleanup and introduction of level-specific targets #599
Conversation
b4aee3c
to
f3e1239
Compare
2001594
to
8bf3698
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Arm Cortex-A76 (Raspberry Pi 5) benchmarks
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
29036 cycles |
29020 cycles |
1.00 |
ML-KEM-512 encaps |
35344 cycles |
35398 cycles |
1.00 |
ML-KEM-512 decaps |
45883 cycles |
45886 cycles |
1.00 |
ML-KEM-768 keypair |
49401 cycles |
49348 cycles |
1.00 |
ML-KEM-768 encaps |
55601 cycles |
55571 cycles |
1.00 |
ML-KEM-768 decaps |
70348 cycles |
70329 cycles |
1.00 |
ML-KEM-1024 keypair |
72071 cycles |
72075 cycles |
1.00 |
ML-KEM-1024 encaps |
80892 cycles |
80847 cycles |
1.00 |
ML-KEM-1024 decaps |
100731 cycles |
100681 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intel Xeon 4th gen (c7i)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
13510 cycles |
13540 cycles |
1.00 |
ML-KEM-512 encaps |
17261 cycles |
17373 cycles |
0.99 |
ML-KEM-512 decaps |
22722 cycles |
22944 cycles |
0.99 |
ML-KEM-768 keypair |
22487 cycles |
22553 cycles |
1.00 |
ML-KEM-768 encaps |
24481 cycles |
24621 cycles |
0.99 |
ML-KEM-768 decaps |
32437 cycles |
32674 cycles |
0.99 |
ML-KEM-1024 keypair |
31449 cycles |
31432 cycles |
1.00 |
ML-KEM-1024 encaps |
35115 cycles |
34946 cycles |
1.00 |
ML-KEM-1024 decaps |
45887 cycles |
45851 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intel Xeon 3rd gen (c6i)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
20345 cycles |
20332 cycles |
1.00 |
ML-KEM-512 encaps |
27012 cycles |
27009 cycles |
1.00 |
ML-KEM-512 decaps |
35816 cycles |
35816 cycles |
1 |
ML-KEM-768 keypair |
34891 cycles |
34890 cycles |
1.00 |
ML-KEM-768 encaps |
38130 cycles |
38186 cycles |
1.00 |
ML-KEM-768 decaps |
50936 cycles |
50935 cycles |
1.00 |
ML-KEM-1024 keypair |
47984 cycles |
47973 cycles |
1.00 |
ML-KEM-1024 encaps |
54144 cycles |
54150 cycles |
1.00 |
ML-KEM-1024 decaps |
71586 cycles |
71716 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AMD EPYC 3rd gen (c6a)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
18143 cycles |
18127 cycles |
1.00 |
ML-KEM-512 encaps |
23188 cycles |
23185 cycles |
1.00 |
ML-KEM-512 decaps |
30612 cycles |
30507 cycles |
1.00 |
ML-KEM-768 keypair |
31072 cycles |
31071 cycles |
1.00 |
ML-KEM-768 encaps |
34171 cycles |
34169 cycles |
1.00 |
ML-KEM-768 decaps |
44758 cycles |
44724 cycles |
1.00 |
ML-KEM-1024 keypair |
44618 cycles |
44607 cycles |
1.00 |
ML-KEM-1024 encaps |
49871 cycles |
49891 cycles |
1.00 |
ML-KEM-1024 decaps |
64347 cycles |
64348 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AMD EPYC 4th gen (c7a)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
15096 cycles |
15077 cycles |
1.00 |
ML-KEM-512 encaps |
19680 cycles |
19664 cycles |
1.00 |
ML-KEM-512 decaps |
26303 cycles |
26317 cycles |
1.00 |
ML-KEM-768 keypair |
25607 cycles |
25547 cycles |
1.00 |
ML-KEM-768 encaps |
28177 cycles |
28119 cycles |
1.00 |
ML-KEM-768 decaps |
37867 cycles |
37839 cycles |
1.00 |
ML-KEM-1024 keypair |
35550 cycles |
35583 cycles |
1.00 |
ML-KEM-1024 encaps |
40956 cycles |
40970 cycles |
1.00 |
ML-KEM-1024 decaps |
53728 cycles |
54481 cycles |
0.99 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intel Xeon 4th gen (c7i) (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
34864 cycles |
34805 cycles |
1.00 |
ML-KEM-512 encaps |
45010 cycles |
44977 cycles |
1.00 |
ML-KEM-512 decaps |
58962 cycles |
58905 cycles |
1.00 |
ML-KEM-768 keypair |
59130 cycles |
59222 cycles |
1.00 |
ML-KEM-768 encaps |
71757 cycles |
71873 cycles |
1.00 |
ML-KEM-768 decaps |
89238 cycles |
89330 cycles |
1.00 |
ML-KEM-1024 keypair |
87591 cycles |
87365 cycles |
1.00 |
ML-KEM-1024 encaps |
104638 cycles |
104579 cycles |
1.00 |
ML-KEM-1024 decaps |
127572 cycles |
127566 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Arm Cortex-A55 (Snapdragon 888) benchmarks
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
58341 cycles |
58368 cycles |
1.00 |
ML-KEM-512 encaps |
65750 cycles |
65797 cycles |
1.00 |
ML-KEM-512 decaps |
84488 cycles |
84531 cycles |
1.00 |
ML-KEM-768 keypair |
98984 cycles |
98990 cycles |
1.00 |
ML-KEM-768 encaps |
110337 cycles |
110464 cycles |
1.00 |
ML-KEM-768 decaps |
136838 cycles |
136532 cycles |
1.00 |
ML-KEM-1024 keypair |
150233 cycles |
150179 cycles |
1.00 |
ML-KEM-1024 encaps |
166349 cycles |
166330 cycles |
1.00 |
ML-KEM-1024 decaps |
202290 cycles |
202544 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graviton3
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
18960 cycles |
18965 cycles |
1.00 |
ML-KEM-512 encaps |
23561 cycles |
23553 cycles |
1.00 |
ML-KEM-512 decaps |
30674 cycles |
30665 cycles |
1.00 |
ML-KEM-768 keypair |
32304 cycles |
32294 cycles |
1.00 |
ML-KEM-768 encaps |
35880 cycles |
35891 cycles |
1.00 |
ML-KEM-768 decaps |
46028 cycles |
46046 cycles |
1.00 |
ML-KEM-1024 keypair |
46623 cycles |
46629 cycles |
1.00 |
ML-KEM-1024 encaps |
52456 cycles |
52444 cycles |
1.00 |
ML-KEM-1024 decaps |
66289 cycles |
66247 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graviton2
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
29047 cycles |
29021 cycles |
1.00 |
ML-KEM-512 encaps |
35361 cycles |
35399 cycles |
1.00 |
ML-KEM-512 decaps |
45888 cycles |
45885 cycles |
1.00 |
ML-KEM-768 keypair |
49397 cycles |
49361 cycles |
1.00 |
ML-KEM-768 encaps |
55610 cycles |
55579 cycles |
1.00 |
ML-KEM-768 decaps |
70375 cycles |
70297 cycles |
1.00 |
ML-KEM-1024 keypair |
72120 cycles |
72009 cycles |
1.00 |
ML-KEM-1024 encaps |
80925 cycles |
80789 cycles |
1.00 |
ML-KEM-1024 decaps |
100794 cycles |
100661 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AMD EPYC 4th gen (c7a) (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
45762 cycles |
45731 cycles |
1.00 |
ML-KEM-512 encaps |
56885 cycles |
56878 cycles |
1.00 |
ML-KEM-512 decaps |
76207 cycles |
76270 cycles |
1.00 |
ML-KEM-768 keypair |
74520 cycles |
74568 cycles |
1.00 |
ML-KEM-768 encaps |
88558 cycles |
88637 cycles |
1.00 |
ML-KEM-768 decaps |
114472 cycles |
114477 cycles |
1.00 |
ML-KEM-1024 keypair |
109353 cycles |
109427 cycles |
1.00 |
ML-KEM-1024 encaps |
127268 cycles |
127513 cycles |
1.00 |
ML-KEM-1024 decaps |
159963 cycles |
160121 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intel Xeon 3rd gen (c6i) (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
56604 cycles |
56624 cycles |
1.00 |
ML-KEM-512 encaps |
69425 cycles |
69431 cycles |
1.00 |
ML-KEM-512 decaps |
91433 cycles |
91368 cycles |
1.00 |
ML-KEM-768 keypair |
91902 cycles |
91844 cycles |
1.00 |
ML-KEM-768 encaps |
107805 cycles |
107860 cycles |
1.00 |
ML-KEM-768 decaps |
136372 cycles |
136350 cycles |
1.00 |
ML-KEM-1024 keypair |
134763 cycles |
134716 cycles |
1.00 |
ML-KEM-1024 encaps |
155314 cycles |
155141 cycles |
1.00 |
ML-KEM-1024 decaps |
191683 cycles |
191643 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AMD EPYC 3rd gen (c6a) (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
52401 cycles |
52178 cycles |
1.00 |
ML-KEM-512 encaps |
65435 cycles |
65771 cycles |
0.99 |
ML-KEM-512 decaps |
88510 cycles |
88404 cycles |
1.00 |
ML-KEM-768 keypair |
84319 cycles |
84745 cycles |
0.99 |
ML-KEM-768 encaps |
102031 cycles |
101806 cycles |
1.00 |
ML-KEM-768 decaps |
131306 cycles |
132081 cycles |
0.99 |
ML-KEM-1024 keypair |
124609 cycles |
124039 cycles |
1.00 |
ML-KEM-1024 encaps |
145096 cycles |
145723 cycles |
1.00 |
ML-KEM-1024 decaps |
182803 cycles |
183725 cycles |
0.99 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graviton3 (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
45397 cycles |
45388 cycles |
1.00 |
ML-KEM-512 encaps |
54220 cycles |
54215 cycles |
1.00 |
ML-KEM-512 decaps |
71155 cycles |
71153 cycles |
1.00 |
ML-KEM-768 keypair |
74829 cycles |
74828 cycles |
1.00 |
ML-KEM-768 encaps |
86075 cycles |
86062 cycles |
1.00 |
ML-KEM-768 decaps |
108673 cycles |
108808 cycles |
1.00 |
ML-KEM-1024 keypair |
111111 cycles |
111124 cycles |
1.00 |
ML-KEM-1024 encaps |
125928 cycles |
125924 cycles |
1.00 |
ML-KEM-1024 decaps |
154585 cycles |
154614 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graviton2 (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
71315 cycles |
71258 cycles |
1.00 |
ML-KEM-512 encaps |
85169 cycles |
85064 cycles |
1.00 |
ML-KEM-512 decaps |
112672 cycles |
112745 cycles |
1.00 |
ML-KEM-768 keypair |
117663 cycles |
117665 cycles |
1.00 |
ML-KEM-768 encaps |
135356 cycles |
135333 cycles |
1.00 |
ML-KEM-768 decaps |
172107 cycles |
171948 cycles |
1.00 |
ML-KEM-1024 keypair |
174689 cycles |
175207 cycles |
1.00 |
ML-KEM-1024 encaps |
196941 cycles |
197231 cycles |
1.00 |
ML-KEM-1024 decaps |
243019 cycles |
243461 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graviton4
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
18132 cycles |
18141 cycles |
1.00 |
ML-KEM-512 encaps |
22146 cycles |
22155 cycles |
1.00 |
ML-KEM-512 decaps |
28773 cycles |
28789 cycles |
1.00 |
ML-KEM-768 keypair |
30566 cycles |
30569 cycles |
1.00 |
ML-KEM-768 encaps |
33652 cycles |
33637 cycles |
1.00 |
ML-KEM-768 decaps |
43172 cycles |
43176 cycles |
1.00 |
ML-KEM-1024 keypair |
44218 cycles |
44211 cycles |
1.00 |
ML-KEM-1024 encaps |
49662 cycles |
49661 cycles |
1.00 |
ML-KEM-1024 decaps |
62640 cycles |
62655 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graviton4 (no-opt)
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
42034 cycles |
41978 cycles |
1.00 |
ML-KEM-512 encaps |
50091 cycles |
50165 cycles |
1.00 |
ML-KEM-512 decaps |
66107 cycles |
66057 cycles |
1.00 |
ML-KEM-768 keypair |
69151 cycles |
69055 cycles |
1.00 |
ML-KEM-768 encaps |
79855 cycles |
79759 cycles |
1.00 |
ML-KEM-768 decaps |
101120 cycles |
101017 cycles |
1.00 |
ML-KEM-1024 keypair |
102207 cycles |
102205 cycles |
1.00 |
ML-KEM-1024 encaps |
117201 cycles |
117195 cycles |
1.00 |
ML-KEM-1024 decaps |
143472 cycles |
143699 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bananapi bpi-f3 benchmarks
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
335030 cycles |
335043 cycles |
1.00 |
ML-KEM-512 encaps |
445667 cycles |
445693 cycles |
1.00 |
ML-KEM-512 decaps |
593678 cycles |
593768 cycles |
1.00 |
ML-KEM-768 keypair |
556111 cycles |
556100 cycles |
1.00 |
ML-KEM-768 encaps |
697923 cycles |
697883 cycles |
1.00 |
ML-KEM-768 decaps |
889662 cycles |
889605 cycles |
1.00 |
ML-KEM-1024 keypair |
821395 cycles |
821199 cycles |
1.00 |
ML-KEM-1024 encaps |
998247 cycles |
998090 cycles |
1.00 |
ML-KEM-1024 decaps |
1230467 cycles |
1230182 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Arm Cortex-A72 (Raspberry Pi 4) benchmarks
Benchmark suite | Current: 8bf3698 | Previous: 169a6c3 | Ratio |
---|---|---|---|
ML-KEM-512 keypair |
51475 cycles |
52125 cycles |
0.99 |
ML-KEM-512 encaps |
58080 cycles |
58732 cycles |
0.99 |
ML-KEM-512 decaps |
74564 cycles |
74611 cycles |
1.00 |
ML-KEM-768 keypair |
88352 cycles |
87684 cycles |
1.01 |
ML-KEM-768 encaps |
96502 cycles |
96005 cycles |
1.01 |
ML-KEM-768 decaps |
120120 cycles |
120180 cycles |
1.00 |
ML-KEM-1024 keypair |
131979 cycles |
132237 cycles |
1.00 |
ML-KEM-1024 encaps |
144500 cycles |
145035 cycles |
1.00 |
ML-KEM-1024 decaps |
175703 cycles |
176330 cycles |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
ad654d9
to
74ae5bf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please rename run_defined
back to check_defined
.
Rest looks good for me. Thank you!
- Use `{func, kat, nistkat, ...}` to build test binaries. - Use `run_{func, kat, nistkat, acvp}` to run and check test binaries. No make targets have yet been introduced to run benchmarks. Signed-off-by: Hanno Becker <[email protected]>
Previously, the `Makefile` exposed goals for building and running tests for all security levels at once. For example, `make func` would build functionality tests for ML-KEM-{512, 768, 1014}, and `make run_func` would run them all. This commit introduces level-specific phony targets `make {,run_}xxx_{512,768,1024}` to target specific security levels. Signed-off-by: Hanno Becker <[email protected]>
Previously, the make targets for {func, kat, nistkat} were unsuitable for running emulated tests since they would attempt to run the binary natively, even when CROSS_PREFIX was passed for cross-compilation. This commit allows emulated execution of cross-built binaries through the environment variable EXEC_WRAPPER. For readability, within the Makefile, the EXEC_WRAPPER variable is abbreviated by W. Signed-off-by: Hanno Becker <[email protected]>
This commit adds make targets `run_bench_{512,768,1024}` to run the benchmark binaries; similar for bench_components. For the all-level targets `run_bench` and `run_bench_components`, the `.WAIT` primitive is used to ensure that benchmarks are not run in parallel. Signed-off-by: Hanno Becker <[email protected]>
Signed-off-by: Hanno Becker <[email protected]>
Signed-off-by: Hanno Becker <[email protected]>
Signed-off-by: Hanno Becker <[email protected]>
74ae5bf
to
f0a03fe
Compare
done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Based on #602
{func, kat, nistkat, acvp, ...}
to build test binaries.run_{func, kat, nistkat, acvp, bench, bench_components}
to run and check test binaries.