Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use API properly with docker? #110

Open
gustavozantut opened this issue Jan 7, 2025 · 0 comments
Open

How to use API properly with docker? #110

gustavozantut opened this issue Jan 7, 2025 · 0 comments

Comments

@gustavozantut
Copy link

i was reading the docs :

Install Python 3.10.11

Download and install [Microsoft] (https://www.microsoft.com/en-us/download/details.aspx?id=57467) MPI. You will be prompted to choose between an exe, which installs the MPI executable, and an msi, which installs the MPI SDK. Download and install both

Install TensorRT-LLM wheel. Please take the wheel(tensorrt_llm-0.9.0-cp310-cp310-win_amd64.whl) from [here](https://github.com/NVIDIA/ChatRTX/tree/release/0.4.0/ChatRTX_APIs/dist).

cd wheel
pip install tensorrt_llm-0.9.0-cp310-cp310-win_amd64.whl --extra-index-url https://pypi.nvidia.com --extra-index-url https://download.pytorch.org/whl/cu121

Download and install 'ngcsdk-3.41.2-py3-none-any.whl' from [here](https://catalog.canary.ngc.nvidia.com/orgs/nvidia/teams/ngc-apps/resources/ngc_sdk/files?version=3.41.2) and install it using the command below. This enables us to download from NGC:

pip install .\ngcsdk-3.41.2-py3-none-any.whl

Download and install the ChatRTX API SDK present in the dist directory

pip install ChatRTX-0.4.0-py3-none-any.whl

Note: This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

I have the Chatrtx installed on my windows that is running a docker container and i want to acess the rtx api via python in the container, how should i follow the instructions? where do i install the pyhon and mpi?
where do i cd to wheel and for what? where do i pip install things, on windows, on container? on both?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant