How to train flux.1-dev inpainting lora?

[thesby]

I want to finetune flux inpainting. I find that diffusers has support flux, but how to finetune inpainting?

[asomoza]

Hi, flux.1-dev its a distilled model so probably its going to be harder to train it for this and also is not that easy. Basically you'll need to modify the model and add more input channels so it accepts the mask as an input and then train it with a very big dataset of image/mask/result examples.

There's a reason why no one is training or releasing the new models with an inpainting version, and probably is that you need a lot more resources to do it and you don't get that much benefit.

[thesby]

https://huggingface.co/ashen0209/Flux-Dev2Pro

Flux-Dev2Pro can be used to solve the problem about distilled model. And masks can be random mask. So how to finetune the inpainting model.

[asomoza]

I didn't tell you that it can't be done, I just told you the reasons why probably no one else is doing it.

How to train inpainting models is something asked here quite a lot, we don't have a training script for that, you have to modify the current one for it if you want to do it with diffusers.

Since you seem to know the solutions for it, I already told you the general gist of it, you'll need to add more input channels and feed the model training with these additional masks for it to learn the inpainting.

We currently don't have the bandwidth to do one and it's not a priority in general, maybe you can ask in the model training repos.

[thesby]

Get it. Thank you for your suggestions.