Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --no-tiles option to write metadata only #259

Merged
merged 2 commits into from
Feb 5, 2025

Conversation

melissalinkert
Copy link
Member

Super quick thought on glencoesoftware/omero-zarr-pixel-buffer#11 (comment), but not thoroughly tested. Mostly just to capture the suggestion from @sbesson so we don't forget.

Copy link
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally managed to to test this PR. The overall output looks good

sbesson@Sebastiens-MacBook-Pro-3 bioformats2raw % ./bioformats2raw-0.10.0-SNAPSHOT/bin/bioformats2raw "test&sizeX=10000&sizeY=20000.fake" test.zarr --no-tiles
sbesson@Sebastiens-MacBook-Pro-3 bioformats2raw % find test.zarr 
test.zarr
test.zarr/.zattrs
test.zarr/.zgroup
test.zarr/0
test.zarr/0/.zattrs
test.zarr/0/.zgroup
test.zarr/0/0
test.zarr/0/0/.zarray
test.zarr/0/7
test.zarr/0/7/.zarray
test.zarr/0/6
test.zarr/0/6/.zarray
test.zarr/0/1
test.zarr/0/1/.zarray
test.zarr/0/4
test.zarr/0/4/.zarray
test.zarr/0/3
test.zarr/0/3/.zarray
test.zarr/0/2
test.zarr/0/2/.zarray
test.zarr/0/5
test.zarr/0/5/.zarray
test.zarr/OME
test.zarr/OME/METADATA.ome.xml
test.zarr/OME/.zattrs
test.zarr/OME/.zgroup
sbesson@Sebastiens-MacBook-Pro-3 bioformats2raw % cat test.zarr/0/0/.zarray 
{
  "chunks" : [ 1, 1, 1, 1024, 1024 ],
  "compressor" : {
    "clevel" : 5,
    "blocksize" : 0,
    "shuffle" : 1,
    "cname" : "lz4",
    "id" : "blosc"
  },
  "dtype" : "|u1",
  "fill_value" : 0,
  "filters" : null,
  "order" : "C",
  "shape" : [ 1, 1, 1, 20000, 10000 ],
  "dimension_separator" : "/",
  "zarr_format" : 2
}%   

And I was able to consume the new API in the context of the original tests and remove the in-place metadata modification without waiting for minutes to generate.

Having the possibility of quickly creating such a spare Zarr datasets feels advantageous I will perform additional testing with different modalities but should this be moved out of draft for a formal review? Minimally, we should at least include some unit tests. In terms of the option name, --no-tiles works for me, alternative names could be --metadata-only or --sparse.

@melissalinkert
Copy link
Member Author

👍 Yes, I can add some tests and then take this out of draft.

@melissalinkert melissalinkert marked this pull request as ready for review January 23, 2025 21:01
@sbesson sbesson merged commit 068c02d into glencoesoftware:master Feb 5, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants