Populate image metadata without allocating memory for the entire image content

[kostrykin]

The num_unique_values metadata element for images was added in #18951. It was later observed that to populate this element, the image was loaded into memory, which required memory allocation in the size of the entire image. In #19760, it was thus proposed to remove the num_unique_values metadata element.

As described here, it is also possible to populate the num_unique_values metadata element without allocating memory for the entire image. There are two orthogonal strategies how this is achieved for TIFF and PNG images in this PR.

A hybrid strategy is employed to extract the necessary information from TIFF files without loading them entirely into memory. TIFF files with multiple segments (aka tiles, stripes) are processed by reading the segments one-by-one. This means that only the memory for a single segment needs to be allocated. TIFF files, however, that have only a single segment, are read by memory-mapping the image data into virtual memory. The image data is then read and processed chunk by chunk. In this case, only the memory of the size of a single chunk needs to be allocated.

In addition, this PR cleans up some terminology. There was confusion regarding pages/series in TIFF files.

For PNG images, the pypng library is used to read and process the image data row-by-row. This should allocate only as much memory as required for a single row of the image. The pypng library is added as a dependency (written entirely in Python, comes without further dependencies, only 58kB in size, see on PyPI).

For images other than TIFF and PNG, the num_unique_values metadata element is currently not computed.

cc @mvdbeek

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[mvdbeek]

Thank you!