Subsample Upscaling Best Practices Question

[awpeoith1254]

Hi - I am not sure if I am using this feature correctly, so I was hoping to clarify, If you have input images that are very high quality (e.g., 4k+) should you use the 'subsample upscale' function (e.g., 256, 512) or should that be avoided in those scenarios because the input is already high quality?

Does that answer change if you have a faceset of high quality images?

[C0untFloyd]

There's a very brief explanation why to use post-processing in the wiki

The face swapper model is limited to a 128x128px output. Even if you input 4096x4096 hires faces, you will get 128x128 pixel out. There is a 512x512 face swapping model but it is behind a paywall and not freely available. So to get high-quality results even with higher resolutions, some workarounds and shortcuts were implemented. Subsample upscale or pixel boost presents the inswapper different slices of the input material, lets it swap into 128x128 boxes and merges them together at the end. This is is still not like the real thing but a huge step forward. Another approach is to use a so called face enhancer/restoration model. This results in very high quality results but the downside is that sometimes certain special details are missing, sometimes stuff which makes a face unique. I think you should should just try whatever fits you the best.

[awpeoith1254]

Hi Count -

I appreciate you taking the time to respond, and I think I understand. Is this right?

• Model is 'capped' at 128x128px so anything about that in terms of input will get disregarded (or i guess downscaled)
• The 256x256px model sub-sample runs the 128x128px process 4x times and the 4 output swaps get combined to mimic 256x
• Same thing for 512x512px but that process is repeated 16x vs. 4x times before the stitching occurs

It's interesting because I mostly try and do videos but they haven't seemed to be noticeably slower, but it must be doing that process on every single frame right so i would have thought it would be 4x, or 16x slower. I guess that is thanks to whatever voodoo magic you have done to accelerate things!

Re: the 512x512 model - it looks like there is no way to try it and pay to have the model to run locally. I guess that makes sense as they are probably worried about piracy but it's a shame.

[C0untFloyd]

Model is 'capped' at 128x128px so anything about that in terms of input will get disregarded (or i guess downscaled)

Yes.

The 256x256px model sub-sample runs the 128x128px process 4x times and the 4 output swaps get combined to mimic 256x

256px = 2 times, 512px 4 times. Horizontal and vertical resolution can be done both in 1 go.

so i would have thought it would be 4x, or 16x slower. I guess that is thanks to whatever voodoo magic you have done to accelerate things!

The Rope guy gets all the credit for the smart idea and the code I've more or less ripped and tweaked to my likings from facefusion. Facefusion because it used numpy instead of pytorch for the slicing and merging which makes this run better in parallel and more compatibel to other gpus. Actually, I never measured this myself but the biggest performance hit is probably the face and landmark detection.

[awpeoith1254]

gotcha very cool - thank you for the explanation!

[Megaderp-del]

Thank you for the explanation, I was wondering the same.

So am I right that I should use either subsample upscale or a restoration model?

Or is the result better If I use both?

[C0untFloyd]

You can use both or none, it's just down to personal preference. You can easily check out the result in the preview. For example I never use the post enhancers because I'd rather have a raw blurry output instead of the perfect, glossy look with the side-effect of losing the likeness but to each their own.