Create GitHub workflow to run benchmark suite

[MilesCranmer]

GitHub actions are desirable for running benchmarks for a variety of reasons:

1. Consistent across runs (see https://labs.quansight.org/blog/2021/08/github-actions-benchmarks/)
2. Can be run on a schedule or on a trigger (e.g., every new Julia tag), without needing someone to run on their own machine
3. Makes it easy to set up various languages, as there is usually a GitHub action that someone has already set up
4. Lets you change versions of these languages easily

This GitHub action successfully runs a subset of the languages and generates the csv data. The workflow runs for the following languages:

• C, fortran, java, javascript, julia, python, R, and rust.

The following benchmarks are not part of the workflow, for the reasons given below:

• Lua, Go (install fine, but the benchmarks themselves are broken)
• Octave (no maintained github action)
• Mathematica, MATLAB (proprietary)

Current result (it prints the csv at the end):

c,iteration_pi_sum,8.028984
c,matrix_multiply,43.012142
c,matrix_statistics,5.007982
c,parse_integers,0.19634
c,print_to_file,20.508051
c,recursion_fibonacci,0.025188
c,recursion_quicksort,0.422955
c,userfunc_mandelbrot,0.08167
fortran,iteration_pi_sum,8.028663
fortran,matrix_multiply,57.163952
fortran,matrix_statistics,8.046266
fortran,parse_integers,0.753935
fortran,print_to_file,113.916496
fortran,recursion_fibonacci,4.4e-5
fortran,recursion_quicksort,0.483927
fortran,userfunc_mandelbrot,7.8e-5
java,iteration_pi_sum,16.370829
java,iteration_sinc_sum,0.049201
java,matrix_multiply,788.083768
java,matrix_statistics,30.276736
java,parse_integers,0.274402
java,print_to_file,99.797282
java,recursion_fibonacci,0.0424
java,recursion_quicksort,1.006608
java,userfunc_mandelbrot,0.136501
javascript,iteration_pi_sum,10.5
javascript,matrix_multiply,2900.0
javascript,matrix_statistics,46.9
javascript,parse_integers,0.64
javascript,print_to_file,118.0
javascript,recursion_fibonacci,0.109
javascript,recursion_quicksort,1.61
javascript,userfunc_mandelbrot,0.149
julia,iteration_pi_sum,8.028063
julia,matrix_multiply,33.387676
julia,matrix_statistics,8.219065
julia,parse_integers,0.137201
julia,print_to_file,18.368588
julia,recursion_fibonacci,0.0482
julia,recursion_quicksort,0.469904
julia,userfunc_mandelbrot,0.0796
python,iteration_pi_sum,630.6591033935547
python,matrix_multiply,49.559593200683594
python,matrix_statistics,51.499128341674805
python,parse_integers,1.6732215881347656
python,print_to_file,54.22806739807129
python,recursion_fibonacci,2.522706985473633
python,recursion_quicksort,11.09170913696289
python,userfunc_mandelbrot,6.908893585205078
r,iteration_pi_sum,236.0
r,matrix_multiply,116.0
r,matrix_statistics,78.0
r,parse_integers,4.0
r,print_to_file,1325.0
r,recursion_fibonacci,10.0
r,recursion_quicksort,22.0
r,userfunc_mandelbrot,20.0
rust,iteration_pi_sum,8.029562
rust,matrix_multiply,46.196557
rust,matrix_statistics,6.52925
rust,parse_integers,0.186271
rust,print_to_file,11.194186
rust,recursion_fibonacci,0.046293
rust,recursion_quicksort,0.428904
rust,userfunc_mandelbrot,0.080522

[MilesCranmer]

Suite doesn't seem to be working on the latest Julia build as libopenblas64_.a is not generated in recent versions. I'm pretty sure those are just used in the perf.h to ensure that the same blas version is used in the C benchmarks.

Any suggestions for what I need to modify @johnfgibson?

Edit: am using the dynamic library for OpenBLAS instead, and linking with -lopenblas.

[MilesCranmer]

The Lua benchmark seems out of date as well.

[MilesCranmer]

The rest seem to be working. Ready for review.

[mkitti]

This is a great contribution. Thank you.

[giordano]

Might want to use https://github.com/JuliaBinaryWrappers/dSFMT_jll.jl/releases/download/dSFMT-v2.2.4%2B2/dSFMT.v2.2.4.x86_64-linux-gnu.tar.gz instead? This is also what's used by Julia itself, so it'd make the comparison fairer.

[MilesCranmer]

Good idea. Will do.

[MilesCranmer]

Actually I can't. This BinaryBuilder version doesn't include the C source code (dSFMT.c), which is required by the benchmark.

It says it's built by the same version though, so I think it's fine?

[MilesCranmer]

Ping for review. Let me know if this is ready for merging.

[MilesCranmer]

Ping for review. @Enet4 @giordano @mkitti do any of you have write access?

[mkitti]

Sorry, I do not have write access.

[MilesCranmer]

@StefanKarpinski @johnfgibson would one of you be willing to approve this? Thanks!

[MilesCranmer]

Thanks, all suggestions implemented.

[DilumAluthge]

Let's give it a try.

[MilesCranmer]

Thanks!!

[mkitti]

Thanks

Thank you for your patience and your contribution, @MilesCranmer .