Reuse search results when given a partially filled directory #98
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mrwnmsr
reviewed
Aug 22, 2024
AustinT
approved these changes
Aug 23, 2024
…otting is enabled
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running search on many targets using preemptible/unreliable compute, it may happen that the run fails having completed a fraction of targets. If such a run is restarted, it will generate a new timestamped output directory and start from scratch. This PR adds a flag
append_timestamp_to_dir, which when set toFalseturns off timestamping, meaning that a restarted run would use the same directory as the old one. On top of this, it implements logic to skip targets which were completed earlier; a target that was partially completed (for example the search statistics were saved but it crashed when dumping the graph) will be purged and recomputed. Finally, the PR also includes a few small tweaks to the surrounding code: adding guards around visualization imports (to make sure search can be ran with the base environment withoutgraphviz) and deleting an__init__.pyfile in an old empty directory.