Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

taichi incompatible with open3d on multiple gpus device #8440

Closed
Jiaviz opened this issue Dec 19, 2023 · 2 comments
Closed

taichi incompatible with open3d on multiple gpus device #8440

Jiaviz opened this issue Dec 19, 2023 · 2 comments

Comments

@Jiaviz
Copy link

Jiaviz commented Dec 19, 2023

Describe the bug
After import open3d, taichi can't initialize multiple cuda devices.
To Reproduce

import taichi as ti
import os
from multiprocessing import Process
import open3d     #no bug if comment this line

@ti.kernel
def do_something():
    for i in range(100000000):
        a = i**2 


def run_taichi_on_gpu(gpu_id):
    # Set the CUDA_VISIBLE_DEVICES environment variable to the desired GPU
    os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu_id)
    
    # Initialize Taichi to run on the specified GPU
    ti.init(arch=ti.cuda)
    
    # Run the Taichi kernel
    do_something()

if __name__ == "__main__":
    # List of GPUs you want to use
    gpus = [0,1,2]  # Example for using GPU 0 and GPU 1
    
    # Create a separate process for each GPU
    processes = [Process(target=run_taichi_on_gpu, args=(gpu_id,)) for gpu_id in gpus]
    
    # Start all processes
    for p in processes:
        p.start()
    
    # Wait for all processes to finish
    for p in processes:
        p.join()

Log/Screenshots
RuntimeError: [cuda_driver.h:operator()@92] CUDA Error CUDA_ERROR_NOT_INITIALIZED: initialization error while calling init (cuInit)

$ python my_sample_code.py
[Taichi] mode=release
[Taichi] version 0.6.29, llvm 10.0.0, commit b63f6663, linux, python 3.8.3
...

Additional comments
If possible, please also consider attaching the output of command ti diagnose. This produces the detailed environment information and hopefully helps us diagnose faster.

If you have local commits (e.g. compile fixes before you reproduce the bug), please make sure you first make a PR to fix the build errors and then report the bug.

@bobcao3
Copy link
Collaborator

bobcao3 commented Dec 21, 2023

it's probably that open3d initialized cuda on its own... What if you init taichi first and then import open3d?

@Jiaviz
Copy link
Author

Jiaviz commented Feb 18, 2024

it's probably that open3d initialized cuda on its own... What if you init taichi first and then import open3d?

Thanks a lot. It works!

@Jiaviz Jiaviz closed this as completed Feb 18, 2024
@github-project-automation github-project-automation bot moved this from Untriaged to Done in Taichi Lang Feb 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

2 participants