Add Docker GPU support

[igor-ritual]

Background & motivation

Docker requires special deployment resources to enable GPU support:

https://docs.docker.com/compose/gpu-support/

Desired behaviour

For Docker, use a new ServiceConfig argument to set the deploy resources.

How important is this to you?

Critical; Kurtosis is unusable for me without it.

What area of the product does this pertain to?

SDK: the Software Development Kit libraries - Typescript, Go, etc.

[leeederek]

Hey @igor-ritual - thanks for filing this! Can you share with me some more details on your workflow (i.e. why you need GPU support)? This information will help us better understand how Kurtosis is being used/examined in the wild and will help us prioritize FRs like this :) Thanks!

[igor-ritual]

Hi @leeederek , I would like to run TGI under kurtosis and the docs show that it needs special cli arguments when run under docker:

docker run --gpus all --shm-size 1g

https://github.com/huggingface/text-generation-inference

[leeederek]

Gotcha - thats useful. I'm acknowledging receipt of this request now and will circle back to you with a decision! One more question: are you running this locally or on a remote server and for how long?

[igor-ritual]

I am running this locally on a Mac for development and remotely on an ubuntu server permanently.