Improve Prompt Matrix

[inukai132]

I think it would be good to change the way the prompt matrix works (or add a separate mode). Currently the matrix is generated with all possible combinations of all of the prompts. IE if your prompt is a|b|c|d|e you get a matrix with entries for

      !b!c  b!c   !bc   bc
!d!e  a     ab    ac    abc
d!e   ad    abd   acd   abcd
!de   ae    abe   ace   abce
de    ade   abde  acde  abcde

So if you have n sub-prompts, you get 2^n outputs.

I think it would be better to have it work so that a(b,c,d) (Using parenthesis here for illustration) makes a matrix like

null a
b    ab
c    ac
d    ad

This could be extended to be able to define up to two groups so that you can do a(b,c)(d,e) to get

      null  b     c
null  a     ab    ac
d     ad    abd   acd
e     ae    abe   ace

So if group 1 is n prompts, and group 2 is m prompts, the matrix is n*m

This would let people test multiple combinations of prompts and have more control over which prompts are combined. For instance, the first group could be different lighting effects, and the second group could be different artist styles. The current solution generates a bunch of combinations that the user might not care about (such as multiple combinations of artist style with no lighting effects), and it scales exponentially instead of linearly.

If the dynamic parameter changes in PR #421 are merged in, this could also be extended to let people generate matrices of parameter values. Maybe put something like cfg-scale (-10,0,30) and/or seed (123,456,789) to generate many prompts with those parameter values, like what is being asked for in #259, #350, or what was shown off in #178.

[BilMorJr]

I can't find the post, but someone else suggested "slots", in which each slot is a list, which I think supports what you want here. So I could say:
an adorable (dog; cat; rabbit) painted by (Norman Rockwell; Francis Barlow; Maxfield Parrish; John James Audobon)
And I would get 12 (3 * 4) permutations.

Other options I would allow

Allow an null item in a list, so someone could have an empty control
A (happy; sad; [null]) tree.
Allow lists which try each option once and tables which try all combinations
A (running; lying; standing) dog ((with a collar; with a waggly tail; with big eyes)).
This would result in the first list, which is muturally exclusive, just being run once each for each item, and the second list to work the way multi-prompts work today, with all the combinations.
I would make a single batch count run all the combinations with one seed, and allow multiple batches so I can get multiple sets of results.

[inukai132]

Played around with this a bit using the existing image grid code and got a basic proof of concept. The following used the prompt a painting of a <forest, desert, swamp, island, plains> painted by <claude monet, greg rutkowski, thomas kinkade>. All images use the same seed so you can compare across them. Still needs work, the position of the text is messed up and the grid itself doesn't get displayed in the UI. Also doesn't change the config options yet.

@BilMorJr I like the idea of having null prompts, I think that can be done pretty easily.

[inukai132]

Example with null prompts, which can be done by adding a comma with no text after it.

a painting of a <forest, desert, swamp, island, plains,> painted by <claude monet, greg rutkowski, thomas kinkade,>

[BilMorJr]

Love it! Exactly what I'm after!

[micwalk]

This exists now but isn't labeled quite the way you have it.

Using this prompt:
@a painting of a (forest|desert|swamp|island|plains) painted by (claude monet|greg rutkowski|thomas kinkade)
Generates this image:

I definitely prefer the labels on the outside vs overlay-ed though. It wouldn't be that hard to change that.

Love the idea of extending to parameter grids!

[osi1880vr]

would it be possible to also allow with a switch or something to change the seed with each iteration?
I was trying something like:
some motive airbrush | airbrush on paper | airbrush on old aged paper | airbrush on concret | airbrush on unprocessed wood | airbrush on furniture | airbrush on transparent glass

But with each iteration I would also like to get a new seed. I understand it is not the usual usecase but it would be great to have a switch like new seed each iteration yes/no :)

Last but not least, I really like your tool, thank you very much for all your effort you are spending!!!

[osi1880vr]

Did you enable the Advanced Option for Matrix Prompt ? Try @(some stuff|other stuff|something else) some great picture idea For me the @ did the Trick, i missed that too in my first tries @hlky maybe in the text oft hat option in the advance tab just add the @ to your example would make things much easier =) Von: tamueller ***@***.***> Gesendet: Samstag, 10. September 2022 11:01 An: sd-webui/stable-diffusion-webui ***@***.***> Cc: osi1880vr ***@***.***>; Comment ***@***.***> Betreff: Re: [sd-webui/stable-diffusion-webui] Improve Prompt Matrix (Discussion #427) Is it somehow possible to generate multiple images with that? e.g. a (cat|dog|bird) wearing a uniform with batch size 3 would generate me 3 cats, 3 birds and three dogs with a uniform? I tried different variations of <cat|dog|bird> (cat|dog|rabbit) in the prompt, but always got catdogs, catbirds and dogbirds as well. — Reply to this email directly, view it on GitHub <#427 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AU2U5IERST37COGSC5P4U4TV5RE6BANCNFSM6AAAAAAQCDZUPE> . You are receiving this because you commented. <https://github.com/notifications/beacon/AU2U5IEAKPOASOLNTN2H6TTV5RE6BA5CNFSM6AAAAAAQCDZUPGWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAG4R5W.gif> Message ID: ***@***.*** ***@***.***> >

[tamueller]

Thanks, I already figured out the @-part, and thus deleted my question here. I agree, that this should be added to the documentation :-)