Loading multiple files spends time in GC

[Semptum]

Hi,
I have to load hundreds of TIFF files into memory. Unfortunately, if I try to simply do something like

img = Matrix{Float32}(undef, (image_size[1]*image_size[2], length(shots)))
for i in shots
    img_name = string("shot", i, ".tiff")
    filename = joinpath(folder_path, img_name)
    img[:,i] .= Float32.(Gray.( @views TiffImages.load(filename)[:] ))
end

The garbage collection runs for more than 90% of the time (according to ProfileView). And to collect 230 files (800x800, 8 bit-depth, monochrome), takes a lot of time (30 seconds). The computer I ran it on doesn't have much in the way of RAM but it shouldn't have been using swap.

So I'm looking for a way to make loading images in-place. That is, directly dumping the contents into an array. Is there a way to do so?

Pardon the poor quality of the code, been more than a year that I haven't used Julia and bad habits set in fast.

Best,
Semptum

[tlnagy]

Why not use mmap=true? Also when you Float32.() and Gray.() you're creating arrays that you're then throwing away. You could use reinterpret

Maybe something like

img[:, i] .= reinterpret(Gray{Float32}, vec(TiffImages.load(filename, mmap = true)))

[Semptum]

Hi,
thanks for the quick reply. I had attempted to use mmap=true but observed no difference. And, absolutely agreed about broadcasting conversions like that, I didn't think reinterpret would work since the image is loaded by default as containing 3 channels, so in my mind directly reinterpreting three identical values into a single one would lead to unpredictable behaviour.

Unfortunately using what you suggested did not fix the problem. While there is a speedup (it's about twice as fast), the time spent in GC (in the load function) still dominates the overall runing time.

PS: Additionally, when trying to use the data, I noticed that the values, while proportional to the correct ones, were insanely big, something like 1.7f38. So somewhere along the lines, reinterpretation does something wrong to the exponent I'm guessing. The problem was not present when doing Float32.(Gray.( ))

[tlnagy]

It's a little difficult to diagnose this remotely, I could help a lot better if you shared a minimal working example that I could run on my end.

And, absolutely agreed about broadcasting conversions like that, I didn't think reinterpret would work since the image is loaded by default as containing 3 channels, so in my mind directly reinterpreting three identical values into a single one would lead to unpredictable behaviour.

This wasn't clear from your code that it was a 3-channel image. Either way, have you tried some of the suggestions for using views from JuliaImages?

PS: Additionally, when trying to use the data, I noticed that the values, while proportional to the correct ones, were insanely big, something like 1.7f38. So somewhere along the lines, reinterpretation does something wrong to the exponent I'm guessing. The problem was not present when doing Float32.(Gray.( ))

Yeah, this is happening because it's likely combining all three channels before doing the reinterpret or something similar.

Unfortunately using what you suggested did not fix the problem. While there is a speedup (it's about twice as fast), the time spent in GC (in the load function) still dominates the overall runing time.

That's interesting. Could you give me the output of

julia> ifds(TiffImages.load(filename, mmap = true)

for one of your images? I need to know what kind of TIFF you are trying to load. TiffImages shouldn't allocate much for most common TIFFs.

[Semptum]

It's a little difficult to diagnose this remotely, I could help a lot better if you shared a minimal working example that I could run on my end.

Yes, I understand, I'll generate one that doesn't involve sending large amounts of data.

This wasn't clear from your code that it was a 3-channel image. Either way, have you tried some of the suggestions for using views from JuliaImages?

I'm sorry, that is my bad. The images are originally monochrome but I post-processed them in Python (fiducial marker detection, couldn't do it in Julia), and apparently the output got converted into a three channel image. I thought it was just a quirk of the library, that it always imported in three channels, should've checked my data first.

Rechecking with the original images, it's very fast. But it's also a different computer (I'm at home now and don't have access to the post-processed images to cross-compare). So I need to do some investigating first to determine whether it's the result of the post-processing or something else.

Sorry for the complications. I will come back here and write update once it's a bit clearer.

[Semptum]

So, I checked and on my work computer (8GB of RAM, enough free RAM to not go to swap), the problem exists whatever files I use, be they post-processed or not.
Actually even loading a single file uses a lot of garbage collection.

I generated an example file filled with just random FixedPointNumbers.N0f16 values and the load command used a lot of garbage collection (the test.tiff file is filled with random values).
ProfileView.@profview img = TiffImages.load("test.tiff")

The IFDs are the following:

 IFD, with tags:
        Tag(IMAGEWIDTH, 1024)
        Tag(IMAGELENGTH, 768)
        Tag(BITSPERSAMPLE, 16)
        Tag(COMPRESSION, COMPRESSION_NONE)
        Tag(PHOTOMETRIC, 1)
        Tag(FILLORDER, 1)
        Tag(STRIPOFFSETS, 8)
        Tag(ORIENTATION, 1)
        Tag(SAMPLESPERPIXEL, 1)
        Tag(STRIPBYTECOUNTS, 1572864)
        Tag(XRESOLUTION, 0x40000000//0x40000000)
        Tag(YRESOLUTION, 0x40000000//0x40000000)
        Tag(RESOLUTIONUNIT, 1)
        Tag(SOFTWARE, "National Instruments...")
        Tag(SAMPLEFORMAT, 1)

So it's something wrong on my work PC, it doesn't happen on my home PC (which is admittedly much more powerful)

[tlnagy]

Huh. Not sure. Based on the IFD information, your TIFF looks pretty standard and should load just fine without any GC. Are both of your machines running the same OS?

EDIT: Could you be hitting this issue? https://discourse.julialang.org/t/very-slow-loading-speed-of-tiff-images/99826/15

[Semptum]

I am indeed on Windows on the work machine (where there is this problem) and on Linux at home (where there is no problem). And yeah, it's slower than MATLAB which I'm trying to replace.
Unfortunately I am very much not used to Windows and don't have a Windows machine at home to really debug this.

In any case, disabling the GC around the call to load does help.
I still need to figure out why GC is still called if I disable it within a function as opposed to within the REPL. But at least it's a known issue.

[tlnagy]

In any case, disabling the GC around the call to load does help.

Does it help or does it resolve the GC issue?

Not sure why we forcibly run the GC on Windows in our load function, which was introduced in #79 https://github.com/tlnagy/TiffImages.jl/blame/0594e42d94929a0b4bcb712985e75559eaab96ec/src/utils.jl#L198 Maybe @timholy or @johnnychen94 remember why we are doing this?

Seems like this is definitely not the solution as it makes TiffImages.load look super bad on Windows.

[Semptum]

Sorry, rewrote the comment a couple of times and didn't re-read it. It completely eliminates the issue.

For some reason it just doesn't work if I do it inside my function.

function open_image_series(folder_path, range; image_size = (800,800))
    img = Matrix{Float32}(undef, (image_size[1]*image_size[2], length(range)))
    k=1
    GC.enable(false)
    for i in range
        filename = shot(folder_path, i)
        @views img[:,k] .= vec(reinterpret(Float32,TiffImages.load(filename, mmap = true))
        k+=1
    end
    GC.enable(true)
    return img
end

This, doesn't work (GC is still used in load). But calling GC.enable(false) from the REPL, then calling open_image_series(...) and then enabling GC, this does work.

But I think this has got nothing to do with the package and is just something about Julia GC I'm not getting. So unless you think otherwise, I'm content with the workaround.