cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THC/THCBlas.cu:411

[Lie-huo]

Hello， when i run selfdeblur_ycbcr.py，there is a mistake，
File "selfdeblur_ycbcr.py", line 121, in
out_x = net(net_input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/tmp/selfdDeblur/networks/common.py", line 22, in forward
inputs.append(module(input))
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/tmp/selfdDeblur/networks/common.py", line 22, in forward
inputs.append(module(input))
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/tmp/selfdDeblur/networks/common.py", line 22, in forward
inputs.append(module(input))
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/user1/anaconda3/envs/pytorch1.1/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/tmp/selfdDeblur/networks/non_local_dot_product.py", line 77, in forward
f = torch.matmul(theta_x, phi_x)
RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THC/THCBlas.cu:411
How can solve this problem?

[csdwren]

It seems to be errors of cuda/cudnn version. I have tested two settings: Pytorch 1.0.0+cuda 9.0+cudnn 7.5 and Pytorch 0.4.0+cuda10.0+cudnn7.5. Both of them work well.