Error building extension '_prroi_pooling' #82
Comments
|
I don't have a 4090 at hand. But this seems to be an issue with the CUDA version. What's the CUDA version you are using? Can you try an update-to-date one? For example, 12.1? |
|
I have tried CUDA 10.1 and 11.6. It is useless. I don't have permission to install CUDA 12.1. |
|
“My environment is GPU 4090, nvcc 10.1, torch 1.7.1+cu110.” Can you use nvcc 11.6 instead? |
|
other error happens: /home/ai1015/anaconda3/envs/pytracking/bin/python /mnt/data/ai1015/pytracking/ltr/run_training.py dimp dimp18 |
|
Ada Lovelace (CUDA 11.8 and later) Unfortunately, you need at least 11.8 to compile CUDA kernels on 4090. https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/ |
|
OK, i will try it. thank you for your patience. If i can install CUDA11.8, i will immediately try it and provide feedback. thank you again. |
|
when i use CUDA11.8, there is new error. (GPU 4090, python3.10)
/home/ai1015/anaconda3/envs/pytracking_py310/bin/python /mnt/data/ai1015/pytracking/ltr/run_training.py dimp dimp18 The above exception was the direct cause of the following exception: Traceback (most recent call last): what is the problem? |
|
nvcc fatal : Value 'c++17' is not defined for option 'std' Your g++ version is too low |
when i run the code pytracking that depends on PrecisePooling, I encounter the error.
first of all, i use git clone from gitee to clone the library to my ubuntus, and I check the code is symbolic link. But I still encounter the error. after showing my error messages, I show my environment as follows.
the whole error messages are as follows:
/home/ai1015/anaconda3/envs/pytracking/bin/python /home/ai1015/pytracking/ltr/run_training.py dimp dimp18
Training: dimp dimp18
WARNING: You are using tensorboardX instead sis you have a too old pytorch version.
No matching checkpoint file found
Using /home/ai1015/.cache/torch_extensions as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /home/ai1015/.cache/torch_extensions/prroi_pooling/build.ninja...
Building extension module prroi_pooling...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=prroi_pooling -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/TH -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/THC -isystem /home/ai1015/anaconda3/envs/pytracking/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /home/ai1015/pytracking/ltr/external/PreciseRoIPooling/pytorch/prroi_pool/src/prroi_pooling_gpu_impl.cu -o prroi_pooling_gpu_impl.cuda.o
FAILED: prroi_pooling_gpu_impl.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=prroi_pooling -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/TH -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/THC -isystem /home/ai1015/anaconda3/envs/pytracking/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /home/ai1015/pytracking/ltr/external/PreciseRoIPooling/pytorch/prroi_pool/src/prroi_pooling_gpu_impl.cu -o prroi_pooling_gpu_impl.cuda.o
nvcc fatal : Unsupported gpu architecture 'compute_86'
[2/3] c++ -MMD -MF prroi_pooling_gpu.o.d -DTORCH_EXTENSION_NAME=_prroi_pooling -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/TH -isystem /home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/include/THC -isystem /home/ai1015/anaconda3/envs/pytracking/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /home/ai1015/pytracking/ltr/external/PreciseRoIPooling/pytorch/prroi_pool/src/prroi_pooling_gpu.c -o prroi_pooling_gpu.o
ninja: build stopped: subcommand failed.
Training crashed at epoch 1
Traceback for the error!
Traceback (most recent call last):
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1539, in _run_ninja_build
env=env)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ai1015/pytracking/ltr/trainers/base_trainer.py", line 70, in train
self.train_epoch()
File "/home/ai1015/pytracking/ltr/trainers/ltr_trainer.py", line 97, in train_epoch
self.cycle_dataset(loader)
File "/home/ai1015/pytracking/ltr/trainers/ltr_trainer.py", line 75, in cycle_dataset
loss, stats = self.actor(data)
File "/home/ai1015/pytracking/ltr/actors/tracking.py", line 28, in call
test_proposals=data['test_proposals'])
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/ai1015/pytracking/ltr/models/tracking/dimpnet.py", line 60, in forward
target_scores = self.classifier(train_feat_clf, test_feat_clf, train_bb, *args, **kwargs)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/ai1015/pytracking/ltr/models/target_classifier/linear_filter.py", line 57, in forward
filter, filter_iter, losses = self.get_filter(train_feat, train_bb, *args, **kwargs)
File "/home/ai1015/pytracking/ltr/models/target_classifier/linear_filter.py", line 94, in get_filter
weights = self.filter_initializer(feat, bb)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/ai1015/pytracking/ltr/models/target_classifier/initializer.py", line 164, in forward
weights = self.filter_pool(feat, bb)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/ai1015/pytracking/ltr/models/target_classifier/initializer.py", line 45, in forward
return self.prroi_pool(feat, roi1)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/ai1015/pytracking/ltr/external/PreciseRoIPooling/pytorch/prroi_pool/prroi_pool.py", line 28, in forward
return prroi_pool2d(features, rois, self.pooled_height, self.pooled_width, self.spatial_scale)
File "/home/ai1015/pytracking/ltr/external/PreciseRoIPooling/pytorch/prroi_pool/functional.py", line 44, in forward
_prroi_pooling = _import_prroi_pooling()
File "/home/ai1015/pytracking/ltr/external/PreciseRoIPooling/pytorch/prroi_pool/functional.py", line 33, in _import_prroi_pooling
verbose=True
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 997, in load
keep_intermediates=keep_intermediates)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1202, in _jit_compile
with_cuda=with_cuda)
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1300, in _write_ninja_file_and_build_library
error_prefix="Error building extension '{}'".format(name))
File "/home/ai1015/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1555, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension '_prroi_pooling'
my environment is GPU 4090, nvcc 10.1, torch 1.7.1+cu110.
I have learn about other issues about the errors.
theirs solutions involve changing the version of torch. But I have try torch 1.7.1 1.7.0 1.13.0 and so on. don't solve the problem.
Because of GPU 4090, my cuda must be 11.0 and higher version. If i try cuda 10.1, I will encounter another error message, CUDA error: no kernel image is available for execution on the device, which is regarding the mismatch between GPU and CUDA.
also, the other solution is about using git clone the PreciseRoIPooling. I have tried it and my code about PreciseRoIPooling is symbolic link.
from asking GPT about my error message, the key is nvcc fatal: Unsupported gpu architecture 'compute_86'.
I try some methods from internet solving the problem, it is useless.
I really don't know what is the problem and how to solve it. does anyone have the same problem and can give some advises? thank you very much.
The text was updated successfully, but these errors were encountered: