Triton test failed when running models with Pytorch backend

[cmpute]

I installed lmdeploy from pip, and there is no cuda compiler in the system. Does triton requires a cuda compiler?

[grimoire]

The answer to "Does triton require a Cuda compiler?" depends on your NVIDIA driver version. Triton has a prepackaged PTXAS, it can be used if you have a compatible driver. Or you need to install cuda manually and set TRITON_PTXAS_PATH.

Your log indicates that you don't have a compatible Cuda runtime. Triton needs to link the runtime library to build a kernel launcher.

[cmpute]

Thanks for the reply! Do I need a system CUDA installation? I have installed a nvcc compiler in a conda environment, but still failed to use triton

[grimoire]

The conda environment should be ok. You can add the cuda runtime (libcuda.so) folder path to LIBRARY_PATH.

[cmpute]

This is a issue for triton, I commented on the official repo here: triton-lang/triton#1617 (comment)