You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First of all, thanks a lot for making this available.
I am using YOWO to train with a custom dataset. I prepared my data in the same format than AVA and used the ava.cfg configuration to run.
It is running OK but very slowly (almost 3 hours for each epoch, with total number of trainable parameters equal to 121381770). The machine where I am running has 4 GPUs but I have several reasons to think that YOWO is not using them:
It makes no difference if I set the variable CUDA_VISIBLE_DEVICES to '0,1,2,3' or '0' or '', the speed is the same
I am monitoring GPU usage with nvidia-smi and most of the time I see that GPU usage is 0
If I check CPU usage, python is using 100% all the time
I added code to main.py to see if the GPUs are accesible and I can confirm that they are. The following
print(f" cuda available {torch.cuda.is_available()}") print(f" current device {torch.cuda.current_device()}") print(f" device 0 {torch.cuda.device(0)}") print(f" device count {torch.cuda.device_count()}")
Produces
cuda available True current device 0 device 0 <torch.cuda.device object at 0x7fdcf4a2ae80> device count 4
My question is, do I need to change something else to make YOWO use the GPUs?
The text was updated successfully, but these errors were encountered:
Hello
First of all, thanks a lot for making this available.
I am using YOWO to train with a custom dataset. I prepared my data in the same format than AVA and used the ava.cfg configuration to run.
It is running OK but very slowly (almost 3 hours for each epoch, with total number of trainable parameters equal to 121381770). The machine where I am running has 4 GPUs but I have several reasons to think that YOWO is not using them:
I added code to main.py to see if the GPUs are accesible and I can confirm that they are. The following
print(f" cuda available {torch.cuda.is_available()}") print(f" current device {torch.cuda.current_device()}") print(f" device 0 {torch.cuda.device(0)}") print(f" device count {torch.cuda.device_count()}")
Produces
cuda available True current device 0 device 0 <torch.cuda.device object at 0x7fdcf4a2ae80> device count 4
My question is, do I need to change something else to make YOWO use the GPUs?
The text was updated successfully, but these errors were encountered: