Allow dtype to be passed to blockwise and reduction functions #321
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tomwhite
force-pushed
the
reduction-dtype-fix
branch
from
2556d1c to
d28db0a
Compare
tomwhite
force-pushed
the
reduction-dtype-fix
branch
from
d28db0a to
84425e7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes a bug where the dtype of the output array in the reduction is not correct. The problem is caused by the fact that extra kwargs to the blockwise function cannot include
dtypesince blockwise itself has a dtype arg. This is solved by addingextra_func_kwargsthat can be used to set dtype on the blockwise function.This was discovered in #187 (comment), since TensorStore is stricter than Zarr about casting when setting a values in an array. (For example, TensorStore fails with
Cannot cast array data from dtype('float64') to dtype('float32') according to the rule 'safe', whereas Zarr will silently cast to float32.)