add runstats utility #611
Merged
add runstats utility #611
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
||
| tbl = Tables.columntable(runs) | ||
|
|
||
| # Arrow.append("runs.arrow", tbl) |
There was a problem hiding this comment.
Maybe remove this comment?
There was a problem hiding this comment.
The way I currently expect it to be used it by making some edits to the script as needed by your application. In some cases you want to re-run it and append runs to the table after e.g. checking out a different commit. In that case you'd want this line.
Good to know. By running from Julia we do have access to many more outputs, which we store here. That was the main reason for writing this in Julia. |
Hofer-Julian
approved these changes
Sep 22, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This addresses a part of #462. It is a way to run and log performance related characteristics into a table. That table can then be used to do further analysis on what the effects of e.g. code changes or solver settings are.
This does not yet hook it up to anything, so there is the possibility for bitrot. But I've had this locally for a while, and I wanted to share it. Even if this doesn't run in TeamCity every week it can be useful for local analysis.
There are many columns, and one row per run, so a small part of the table looks like:
Columns are about solver settings, host machine, julia version, git commit, timing and GC.