ControlNet support

[R4rum]

The ControlNet extension improves the control over the composition by a lot. I would imagine it being a worthwhile addition.

Additional context
Currently the extension even throws errors when working in Krita:

Error running process: stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py
Traceback (most recent call last):
  File "stable-diffusion-webui/modules/scripts.py", line 386, in process
    script.process(p, *script_args)
  File "stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 390, in process
    enabled, module, model, weight, image, scribble_mode, \
ValueError: not enough values to unpack (expected 13, got 0)

[Lute-4]

It would be very useful! :)

Here is the link to the webui extension:
https://github.com/Mikubill/sd-webui-controlnet

[Lute-4]

Maybe useful, "added an API layer":
Mikubill/sd-webui-controlnet#194

[ethanfel]

The ControlNet extension improves the control over the composition by a lot. I would imagine it being a worthwhile addition.

Additional context Currently the extension even throws errors when working in Krita:

Error running process: stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py
Traceback (most recent call last):
  File "stable-diffusion-webui/modules/scripts.py", line 386, in process
    script.process(p, *script_args)
  File "stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 390, in process
    enabled, module, model, weight, image, scribble_mode, \
ValueError: not enough values to unpack (expected 13, got 0)

This is more an issue of controlnet than the plugin, seems the controlnet is interfering even when disabled.

Regarding it's support, yeah it would be amazing. I'm back to 2 A111 because of it. I've bind every folder with models so it's just the setting and the extensions, it's not a big deal. Controlnet is the best improvement since local fine tuning.

[Interpause]

Seems complex enough that I can't just guess what is going wrong, I may take a look at it if I have time.

[Lute-4]

This excellent free and open source photoshop plugin just had a new update with controlnet, maybe the code of this plugin can help the one here for krita :
https://www.reddit.com/r/StableDiffusion/comments/11iuqhv/major_update_automatic1111_photoshop_stable/
https://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiffusion-Plugin
AbdullahAlfaraj/Auto-Photoshop-StableDiffusion-Plugin#124

[Interpause]

I see, maybe I will look at it when my internship is over.

[JasonS09]

I'm currently working on a personal ControlNet implementation on my end. I'll inform how things turn out.

[Sugarun]

i tried to implement controlnet to auto-sd and use it in krita but apparently i'm too stupid for that. if you could describe how you did it i would be eternally grateful.

[JasonS09]

i tried to implement controlnet to auto-sd and use it in krita but apparently i'm too stupid for that. if you could describe how you did it i would be eternally grateful.

I haven't done it yet, I'm still working on the UI but I've got some approaches in mind. If it turns great then I'll share it.

[dmarx]

the deforum plugin has good controlnet support plus a lot of other features (especially powerful for image-to-image animation). if getting controlnet support added is going to require onboarding another plugin, might get more bang for your buck integrating deforum. no idea if there's a difference in LOE between adding support for just controlnet vs. deforum. maybe if we reach out to the gradio team, they could come up with some more generic support between gradio applications and krita?

[Interpause]

the gradio team would probably not do that...