50gb joins #61
50gb joins #61
Conversation
| @@ -0,0 +1 @@ | |||
| 13.0.0.1 | |||
There was a problem hiding this comment.
I don't think those VERSION files should be part of git tree as they are very dynamic, and keeping them updated will be just polluting git history
| @@ -881,3 +881,21 @@ ip-172-31-31-147,1698326993,collapse,2.0.3,,join,J1_1e8_NA_0_1,1698348637.08848, | |||
| ip-172-31-31-147,1698326993,collapse,2.0.3,,join,J1_1e8_NA_0_1,1698349833.44723,finish,0,0 | |||
| ip-172-31-31-147,1698326993,collapse,2.0.3,,join,J1_1e9_NA_0_0,1698349848.46256,start,, | |||
| ip-172-31-31-147,1698326993,collapse,2.0.3,,join,J1_1e9_NA_0_0,1698370827.50361,finish,0,0 | |||
| ip-172-31-31-147,1699266597,data.table,1.14.9,88039186915028ab3c93ccfd8e22c0d1c3534b1a,join,J1_1e9_NA_0_0,1699266602.09238,start,, | |||
There was a problem hiding this comment.
same logs.csv and time.csv. What I believe makes more sense is to have those 'artifacts' is a gh-pages (or any other non-master) branch and keep master branch focused on the code only.
But feel free to ignore :) it is just how that was being handled in the past and just feels like best practices.
There was a problem hiding this comment.
The version and time.csv/logs.csv have been included to verify that others have run the benchmark. The readme now has a new PR process for whenever someone wants to update the benchmark, which requires people to run the benchmark themselves.
If new results are produced because a solution has a new version, then I would like to see the version bump in the VERSION file as well. Chances are I will ignore any bug fix versions. I agree that the VERSION files can become polluted though, I may look into a different solution for that.
As seen many of these solutions do not complete the joins. the timeout is 240 minutes, which is 4 hours. The timeout is time required for loading the data and executing all queries.
Also adding a couple of version files that seem to have been missed.