Bug/memory loss #126
Closed
Bug/memory loss #126
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Handle errors more gracefully
# Conflicts: # cerulean_cloud/cloud_function_ais_analysis/main.py # cerulean_cloud/cloud_run_orchestrator/handler.py # cerulean_cloud/database_client.py # cerulean_cloud/models.py
🍹
|
🍹
|
sstill88
reviewed
Dec 11, 2024
| features=tileset_fc_list, min_overlaps_to_keep=1 | ||
| ) | ||
|
|
||
| # Stitch inferences |
There was a problem hiding this comment.
Is this stitching and ensembling done twice or is there a subtle difference I am missing?
There was a problem hiding this comment.
Stitching and Ensembling are related concepts that we try to keep distinct in this code. Here's my working definition...
Stitching: connecting adjacent tiles that do not overlap
Ensembling: combining multiple (overlapping) runs of a collection of data into a single more robust result
I'm not sure if I am interpreting your question exactly correctly, but yes we do run nms_feature_reduction() twice--
• the first takes place inside postprocess_tileset() and is applied to the scene after having gone through Stitching
self.nms_feature_reduction(feature_collection)
• the second takes place in _orchestrate() and is our primary method of Ensembling
model.nms_feature_reduction(features=tileset_fc_list, min_overlaps_to_keep=1)
This operation goes like N^2, so smaller datasets lead to substantially fewer intersection operations.
The first one is used to generate a single tileset's best candidates for the oil in the scene, primarily used to eliminate the lowest likelihood classes if a single polygon of oil shows up as in two or more class layers (very likely).
The second one is used to choose which of the tileset's results is the best.
Is your question / argument that we might be able to do a single nms_feature_reduction? If so, I'm not sure that we would have much to gain by that, and possibly suffer much worse performance on some "messy" scenes where the polygon count is high.
There was a problem hiding this comment.
This comment is referring to the stitching and ensemble blocks each being run twice
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Feel free to merge MAIN back into this branch before running your review.