Running auto3dseg_hello_world.py triggers a multi-processing error on multi-GPU cluster

[dvitaly07]

Running auto3dseg_hello_world.py triggers a multi-processing error on multi-GPU cluster, please advise:

[KumoLiu]

Hi @dvitaly07, thanks for your interest here.
The Auto3DSeg modules support multiple GPU since MONAI 1.2. The GPU devices are selected via setting the environment variable CUDA_VISIBLE_DEVICES.
https://github.com/Project-MONAI/tutorials/blob/main/auto3dseg/notebooks/auto_runner.ipynb

[dvitaly07]

Hi, unfortunately setting the CUDA_VISIBLE_DEVICES environment variable did not resolve the issue. I am still getting the same exact error message. Please advise.

[KumoLiu]

I don't know which notebook do you refer to.
I don't think all the part in this notebook can support multigpu such as prepare data, you need modify it by yourself.
You can refer to this folder for more details: https://github.com/Project-MONAI/tutorials/tree/main/acceleration/distributed_training

Thanks.

[dvitaly07]

Hi, unfortunately setting the CUDA_VISIBLE_DEVICES environment variable did not resolve the issue. I am still getting the same exact error message. Please advise. On Monday, August 19, 2024 at 12:04:22 AM EDT, YunLiu ***@***.***> wrote: Hi @dvitaly07, thanks for your interest here. The Auto3DSeg modules support multiple GPU since MONAI 1.2. The GPU devices are selected via setting the environment variable CUDA_VISIBLE_DEVICES. https://github.com/Project-MONAI/tutorials/blob/main/auto3dseg/notebooks/auto_runner.ipynb — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>