Fix of nodata, fill value, mask and data type propagation in EOdal #78
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The previous dtype checking function did not work well and missed a lot of edge cases. Now, it should work with all `int`, `uint`, `float` and `complex` data types
a TypeError will be raised if the data type of the band data is incompatible with the data type of the no-data value (e.g., nan is used for an integer array).
Codecov Report
@@ Coverage Diff @@
## master #78 +/- ##
==========================================
+ Coverage 56.81% 56.94% +0.12%
==========================================
Files 67 68 +1
Lines 5220 5265 +45
Branches 1047 1059 +12
==========================================
+ Hits 2966 2998 +32
- Misses 1957 1960 +3
- Partials 297 307 +10
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses #77 that shed some light on masking issues in EOdal. In particular, the mask of the Sentinel-2 Scene Classification layer was not propagated correctly. While debugging this issue, I found some general problems with the propagation of data types, fill and nodata values caused by inconsistencies in the code and poor type checking.
The major fixes include:
np.ma.masked_array. If not (previous behavior) numpy would interfere a fill value that would often not be the no-data type of the raster. In the forthcoming release of EOdal, the fill-value will now equal the nodata value.RasterCollectionhas been improved and extended to allINT,FLOAT, andCOMPLEXdata types currently supported by numpy. In the previous version, all data types were cast tonumpy.float32or evennumpy.float64leading to an unnecessary high consumption of memory.scaleandoffsetparameters are now correctly written to the file metadata when exporting aBandorRasterCollectionobject. In previous versions of EOdal, the information about this two attributes was not written to output. Also, the way scale and offset are applied, is now consistent withQGISmeaning that raster files created by EOdal will be shown with correct scale and offset in QGIS (QGIS applies them on the fly when displaying the raster data)In addition, a new feature was implemented allowing to create
cloud-optimized GeoTiffs (COG)fromRasterCollectionobjects. A new keyword-argumentas_coghas been added toRasterCollection.to_rasterio.as_cogis False by default. If True, a COG will be generated.