Enabling AMD GPU

[Moohsticks]

Hello there,

I was wondering if someone could give me a ltitle help on enabling a AMD GPU?

In the installation guide it says the following 'pip install onnxruntime-directml==1.15.1' but I couldn't install that version so I grabbed 1.17.0 and that installed just fine. The issue I'm having is that I can't seem to figure out how to see it as a provider.

Any ideas?

Thanks!

[C0untFloyd]

Some months ago this did work for AMD people:
#186 (comment)

[Moohsticks]

Thanks, I will keep in mind the thread count when I manage to get dml to show up as a provider

[thegreek1990]

are you using the python run in the main folders?

in main folder, type cmd
copy and paste command when starting roop

python run.py --execution-provider dml

also, after you run the webinstaller. sometimes its quirky. ie I have stable diffusion also and it just spreads chaos with directml
in the main folder, try pip install -r requirements.txt

then try above with installing directml etc etc

[Moohsticks]

Okay so that got us in and loaded up the dml option as a provider, I just went ahead to test it and it threw up an error

I did change the settings, apply them and restart the whole thing

[thegreek1990]

noooo not the dreadful face not found error.

https://huggingface.co/ezioruan/inswapper_128.onnx/tree/main

https://huggingface.co/ezioruan/inswapper_128.onnx/resolve/main/inswapper_128.onnx?download=true

it must have messed up and all that fun jazz.

quickfix.

go ahead and download, place in your roop\models folder

[Moohsticks]

That makes sense, it was probably that weird stuck/hanging download I got, since it was that file it happened with.

I replaced the file and I can process without the error now, it's not utilizing my GPU and seems to still be on the CPU

Probably something to do with this still saying Device:cpu?

[thegreek1990]

It's ok it'll say CPU.

as long as it said DML under settings when you first load it up it'll work. Have fun! just make sure thread count is on 2.

[zhafrannr]

can you help me? i stuck on ModuleNotFoundError: No module named 'torch'
i've tried pip install -r requirements.txt, but it didn't work

[thegreek1990]

[Moohsticks]

I have those settings, which I think besides the dml provider is default

This is the processing status and my GPU is sitting at 1% throughout

[aldemoth]

It shouldn't use cpu when provider is not CpuExecutionProvider. Are you sure gpu utilization is %1? If you are using task manager it doesn't show utilization correctly, check radeon software or hwinfo.

[thegreek1990]

Try to change thread count to 2 and restart everything.

…

On Tue, Feb 6, 2024 at 5:09 PM Moohsticks ***@***.***> wrote: I have those settings, which I think besides the dml provider is default Untitled7.png (view on web) <https://github.com/C0untFloyd/roop-unleashed/assets/156854093/d22f153d-12fd-4aa5-bc77-4b357df90c9e> This is the processing status and my GPU is sitting at 1% throughout — Reply to this email directly, view it on GitHub <#428 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BEOAOJUTZ63PTWCN2AQMUJTYSKZ2DAVCNFSM6AAAAABC2H6YOWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DGOBYHE3DM> . You are receiving this because you commented.Message ID: ***@***.*** com>

[Moohsticks]

I've just tried that and a bunch of other settings to see if it made any different but still seems to be the same

[thegreek1990]

Ok give me a few hours to look into it. Theres a way to force the run with the settings with certain commands but I can’t locate it at the moment

…

On Tue, Feb 6, 2024 at 5:38 PM Moohsticks ***@***.***> wrote: I've just tried that and a bunch of other settings to see if it made any different but still seems to be the same — Reply to this email directly, view it on GitHub <#428 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BEOAOJUU3WNKAO5C6WZ7P5DYSK5IRAVCNFSM6AAAAABC2H6YOWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DGOBZGE3DQ> . You are receiving this because you commented.Message ID: ***@***.*** com>

[Moohsticks]

No worries, I will continue to have a mess around with things to see if anything helps and it's something silly. Thank you for the help today, at least we got it to change the provider to dml and run!

[thegreek1990]

Quick question while I’m looking. Which gpu model do you have?

…

On Tue, Feb 6, 2024 at 5:45 PM Moohsticks ***@***.***> wrote: No worries, I will continue to have a mess around with things to see if anything helps and it's something silly. Thank you for the help today, at least we got it to change the provider to dml and run! — Reply to this email directly, view it on GitHub <#428 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BEOAOJUUBCSGHQJG5LN3LQTYSK6ABAVCNFSM6AAAAABC2H6YOWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DGOBZGIYTC> . You are receiving this because you commented.Message ID: ***@***.*** com>

[Moohsticks]

I have a 7800XT

[thegreek1990]

Well I can’t find it but it should be working. I went ahead and reinstalled everything on my end just to see and it’s working. Now that you have the proper python and proper visual studio c++ Wait you do have visual studio right?

…

On Tue, Feb 6, 2024 at 7:14 PM Moohsticks ***@***.***> wrote: I have a 7800XT — Reply to this email directly, view it on GitHub <#428 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BEOAOJVJJ7VK4PSIWNXC5BDYSLIPDAVCNFSM6AAAAABC2H6YOWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DGOBZG44TE> . You are receiving this because you commented.Message ID: ***@***.*** com>

[Moohsticks]

Yeah I do have VS installed, everything should be installed properly so I have no idea, kind of a mystery

[thegreek1990]

Best bet would be to go ahead and delete folder since you had the wrong python before. It’s done this to me before. When I installed stable diffusion is changed something so I had to go and redo everything. Install web first, then install with pip requirements and call it a night. I’ll keep searching though

…

On Tue, Feb 6, 2024 at 8:21 PM Moohsticks ***@***.***> wrote: Yeah I do have VS installed, everything should be installed properly so I have no idea, kind of a mystery — Reply to this email directly, view it on GitHub <#428 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BEOAOJRLRNR3OE22AWJ32ODYSLQMDAVCNFSM6AAAAABC2H6YOWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DGOJQGE2DI> . You are receiving this because you commented.Message ID: ***@***.*** com>

[Moohsticks]

That is a good idea, just going to double check what you mean by 'install web first' to make sure I've got everything right.

But I will give that a go and see what happens

[thegreek1990]

Oh just the installer.bat first for the ffmpeg, etc and then the pip install for anything left. Let me know how it goes

…

On Tue, Feb 6, 2024 at 8:32 PM Moohsticks ***@***.***> wrote: That is a good idea, just going to double check what you mean by 'install web first' to make sure I've got everything right. But I will give that a go and see what happens — Reply to this email directly, view it on GitHub <#428 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BEOAOJUOVLWYPYDK73O7Y3TYSLRVJAVCNFSM6AAAAABC2H6YOWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DGOJQGIZDG> . You are receiving this because you commented.Message ID: ***@***.*** com>

[thegreek1990]

Try one more thing, im still looking for that command.

instead of typing cmd and entering the start command thru windows console


. Just blank out the main folder and enter the cmd to try with python instead

it's worth a shot.

[Moohsticks]

Just got back today and tried this, still the same but it's working either way so who knows, maybe it just doesn't work too well with the 7000 series AMD cards.

[C0untFloyd]

The commandline args were removed a long time ago and won't work. Perhaps I found a bug:
https://github.com/C0untFloyd/roop-unleashed/blob/main/roop/utilities.py#L205
This returns the cpu as a device if no cuda or mac provider is found. Perhaps this needs to be added:

    if 'DmlExecutionProvider' == prov:
        return 'dml'
    return 'cpu'

I don't have a AMD card, could somebody verify this?

[aldemoth]

I tried that when trying to fix DMDNet but it throws this error:

RuntimeError: Expected one of cpu, cuda, ipu, xpu, mkldnn, opengl, opencl, ideep, hip, ve, fpga, ort, xla, lazy, vulkan, mps, meta, hpu, mtia, privateuseone device type at start of device string: dml

I don't know if we need torch-directml package for that.

My gpu utilization looks low in task manager but it isn't, so I can't comment on author's issue.

Note: I installed torch==2.1.2 instead of torch==2.1.2+cu118 to save bandwidth. I don't know if cuda package supports dml as torch device.

[C0untFloyd]

Thanks for testing, no the CUDA package won't support dml. What about using vulkan then? Wouldn't that work?

[C0untFloyd]

Answering myself, it seems to be impossible to have Torch use DirectML. However Torch is used only in masking and DMDNet Postprocessing meaning it's less important. I was missing RocML and OpenVino though, which I just now added to the device selection.
Btw. can't AMD users just fake CUDA Support with RoCML?

[thegreek1990]

The only thing I remember seeing is the new python isn't compatible with torch when messing around with AMD diffusion "AUTOMATIC1111", but he said he uninstalled the Newest version of Python and then installed the older one. Could that cause some sort of conflict having some newer stuff left over somewhere?

https://learn.microsoft.com/en-us/windows/ai/directml/gpu-pytorch-windows
https://pytorch.org/get-started/locally/

fingers crossed with RoCML in the future.

[salvagedimacaroni]

Hey all,
The past day I have struggling also to make use of DML on RX6600.

I can launch roop and it shows this on top: python: 3.9.18 • torch: 2.2.1+cpu • gradio: 3.48.0
In the settings the provider appears empty but I can use the dropdown.
There is no difference if I choose DML or CPU and make restart, the actual process always fails.

I cannot make sense of this, can somebody suggest something?

[lysxelapsed]

Is ffmpeg installed and added to PATH in your environment variables (Step 3)?

[Stanley154]

D:\a_work\1\s\onnxruntime\core\providers\cuda_call.cc:121 onnxruntime::CudaCall D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDA failure 35: CUDA driver version is insufficient for CUDA runtime version ; GPU=-15163856 ; hostname=LAPTOP-CDGC5 ; file=D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=245 ; expr=cudaSetDeice(info_.device_id);

Hey guys, I get this error when running roop unleashed, I use an amd gpu and I am not very much into technical stuff, can anybody please help me out, I need this software to work urgently, a detailed explanation would be much apppreciated
Thank you

[Stanley154]

pip install -r requirements.txt

Plus when I run this command, I got something called error:exception:
Traceback (most recent call last): at the last

[Stanley154]

ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: D:\a_work\2\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\FusedGraphKernel.cpp(70)\onnxruntime_pybind11_state.pyd!00007FFC9D8C8971: (caller: 0007FFC9D8C64C9) Exception(9) tid(1dd8) 8007000E Not enough memory resources are available to complete this operation.

An update guys, now I am getting this error, please help

[C0untFloyd]

Your Laptop AMD gpu doesn't have the necessary memory. 6 Gb is the absolute minimum, try switching to cpu instead using your gpu.

[Stanley154]

Your Laptop AMD gpu doesn't have the necessary memory. 6 Gb is the absolute minimum, try switching to cpu instead using your gpu.

\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDA failure 35: CUDA driver version is insufficient for CUDA runtime version ; GPU=12101484 ; hostname=LAPTOP-CDUFMD5 ; file=D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=245 ; expr=cudaSetDevice(info_.device_id);

When I tried running with cpu, it's showing me this error

[billbalm]

This worked for me, but it runs purely on CPU:

Note: I'm a developer, but complete ignorant on python

1. Installed ffmpeg and put it in the PATH
2. Installed Visual Studio Community 2022, selected C++ in Desktop and Game Development
3. Installed Python 3.11.9 (on 3.12 pytorch fails)
4. Downloaded the source: https://github.com/C0untFloyd/roop-unleashed/archive/refs/tags/4.2.3.zip
5. Unzipped it and renamed the folder to roop-unleashed (probably irrelevant)
6. Opened a Command Prompt and cd'ed into that folder
7. Ran pip install -r ./requirements.txt
8. Edited file settings.py and changed 'cuda' to 'cpu' in self.provider = self.default_get(data, 'provider', 'cuda')
9. Ran python run.py

As it's not running on my GPU (AMD RX 6750 XT) it's extremely slow.

To run on AMD GPU it would require torch with ROCm no? Well, that's not supported on Windows. But appears to be supported on WSL.

However my GPU isn't supported, even on Linux, according to AMD. See https://rocm.docs.amd.com/projects/radeon/en/latest/docs/compatibility/wsl/wsl_compatibility.html

That said, maybe on WSL and a 7900 XT(X), this could work:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.1

[C0untFloyd]

Did you even read some of the previous entries? For AMD DirectML seems to be the way to go and it's as easy as
pip uninstall onnxruntime-gpu pip install onnxruntime-directml==1.15.1

Perhaps there is a newer version already but 1.15.1 worked in the past.

There is #186 (comment) and https://github.com/C0untFloyd/roop-unleashed/wiki/Installation

Using windows I would recommend using the installer https://github.com/C0untFloyd/roop-unleashed/releases/download/4.2.3/roop-unleashed_win4.2.3_installer.zip and after installation change the onnxruntime to directml (like above)

If you don't like installers, you should use git to clone the repo and do a git pull from time to time get the latest changes.

[billbalm]

I did read some. Anyway, thanks for the info. And it worked with latest onnxruntime-directml (1.19.2), in case it's useful to know.

pip uninstall onnxruntime-gpu
pip install onnxruntime-directml

And set provider: dml in config.yaml.

My step by step above was still necessary to install it and run it the first time. Step by step provided by other people here caused errors to me.

[billbalm]

Btw, cpu vs dml in my system (Ryzen 9 3900X / RX 6750 XT) and use case, dml was around 8x faster.