You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The JVM's default max heap varies by OpenJDK version, but is typically rather small compared to the size of memory on a computer. For example, I ran out of memory trying to perform a filter.gauss op on a 1G dataset. After increasing my max heap to 16G by adding -Xmx16g to the additional JVM args area, I was able to perform the operation.
But users will not know to do this, especially Python users who don't know Java. Fiji tries to be kind and autoset the max heap to 3/4 of your physical RAM by default, and I think napari-imagej should do the same. I also think the settings dialog should expose an option directly to control max memory, rather than instructing users in the documentation (which many people don't read, of course) to add the arcane -Xmx... flag to extra JVM args.
Detecting physical memory size in Python is simple with psutil:
As an aside: it's unfortunate that the backing ImgLib2 implementation for filter.gauss seems to scale badly in RAM with the size of the image. In my story above, I was performing the computation on an image wrapped from Python, to an image wrapped from Python. The way it ran out of memory was actually due to the creation of too many and/or large intermediate data objects during the computation, which for something like gauss is surprising given that the amount of intermediate RAM needed should be bounded by something like threads x neighborhood_pixel_count x bytes_per_pixel. My sigmas were (5, 5, 0, 0) so the neighborhood_pixel_count is only 11 x 11, and I have 24 cores according to Java, so that's a measly 12KB of intermediate storage needed for a naive gauss implementation.
Just so that it's logged somewhere on GitHub, here is the OOM stack trace:
OutOfMemoryError
[ERROR] Command errored: filter.gauss
java.lang.OutOfMemoryError: Java heap space
at net.imglib2.img.basictypeaccess.array.AbstractFloatArray.<init>(AbstractFloatArray.java:50)
at net.imglib2.img.basictypeaccess.array.FloatArray.<init>(FloatArray.java:47)
at net.imglib2.img.basictypeaccess.array.FloatArray.createArray(FloatArray.java:58)
at net.imglib2.img.basictypeaccess.array.FloatArray.createArray(FloatArray.java:43)
at net.imglib2.img.cell.CellImgFactory.create(CellImgFactory.java:146)
at net.imglib2.img.cell.CellImgFactory.create(CellImgFactory.java:103)
at net.imglib2.img.cell.CellImgFactory.create(CellImgFactory.java:59)
at net.imglib2.algorithm.gauss3.SeparableSymmetricConvolution.convolve(SeparableSymmetricConvolution.java:293)
at net.imglib2.algorithm.gauss3.SeparableSymmetricConvolution.convolveRealTypeFloat(SeparableSymmetricConvolution.java:144)
at net.imglib2.algorithm.gauss3.SeparableSymmetricConvolution.convolve(SeparableSymmetricConvolution.java:125)
at net.imagej.ops.filter.gauss.DefaultGaussRAI.compute(DefaultGaussRAI.java:87)
at net.imagej.ops.filter.gauss.DefaultGaussRAI.compute(DefaultGaussRAI.java:58)
at net.imagej.ops.special.hybrid.UnaryHybridCF.run(UnaryHybridCF.java:75)
at net.imagej.ops.special.hybrid.UnaryHybridCF.run(UnaryHybridCF.java:97)
at org.scijava.command.CommandModule.run(CommandModule.java:196)
at net.imagej.ops.OpEnvironment.run(OpEnvironment.java:960)
at net.imagej.ops.OpEnvironment.run(OpEnvironment.java:136)
at net.imagej.ops.OpListingModule.run(OpListingModule.java:68)
at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:125)
at org.scijava.module.ModuleRunner.call(ModuleRunner.java:64)
at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:247)
at org.scijava.thread.DefaultThreadService$$Lambda$354/1466391661.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
The JVM's default max heap varies by OpenJDK version, but is typically rather small compared to the size of memory on a computer. For example, I ran out of memory trying to perform a
filter.gauss
op on a 1G dataset. After increasing my max heap to 16G by adding-Xmx16g
to the additional JVM args area, I was able to perform the operation.But users will not know to do this, especially Python users who don't know Java. Fiji tries to be kind and autoset the max heap to 3/4 of your physical RAM by default, and I think napari-imagej should do the same. I also think the settings dialog should expose an option directly to control max memory, rather than instructing users in the documentation (which many people don't read, of course) to add the arcane
-Xmx...
flag to extra JVM args.Detecting physical memory size in Python is simple with
psutil
:The text was updated successfully, but these errors were encountered: