Writing a PermutationSpec for SDXL

[AI-Casanova]

Do you have any plans to Develop a PermutationSpec for SDXL?

If not, would you be so kind as to explain a bit more how you arrived at the one for 1.5?

I read through your conversation over on the original repo, but I'm having trouble following your work compared to a raw model.

[ogkalu2]

I plan to write for XL. I'm just a bit swamped at the moment.

[AI-Casanova]

I think I got the hang of it.

I'm working up a PermutationSpec for SDXL without VAE keys (I find no reason to ever merge VAE, simply replace if necessary) if I get it to work, I'll link it here for your use.

[AI-Casanova]

https://github.com/vladmandic/automatic/blob/dev/modules/merging/merge_PermSpec_SDXL.py

SDXL PermutationSpec

[ogkalu2]

Nicely done! Did you run a test ? Do you know what layers change when you attempt a merge ?

[AI-Casanova]

It seems to work fine (my code is based on @s1dlx code which is based on yours) but I haven't delved deeper.

[AI-Casanova]

[6DammK9]

Finally got it done.
https://github.com/6DammK9/Merge-Stable-Diffusion-models-without-distortion/blob/main/merge_PermSpec_SDXL.py

You need to "sew" it properly, the permutation loop somehow iterlate the whole model in sequence, and checking tensor sizes. No wonder why it is left untouched for months.

Sample image in my fork, with comparasion.

[AI-Casanova]

Amazing! I got it as far as I could, glad someone picked it up from there.