Request - Add support for StyleGanV2

[ofirkris]

Hi, Would be great if you can make your encoder support the latest Stylegan version - Reference to Puzer StyleganV2 fork - https://github.com/rolux/stylegan2encoder

[pbaylies]

@ofirkris I've thought about this - this would be better done as a fork of StyleGAN2. Anyone would be welcome to do this in a new repo, currently I'm taking a break from active development over the holidays.

[oneiroid]

Karras implemented projecting to latent space right there in the original repo yo. Don't think it can be improved any more. He's using tiled (1, 512) dlatent though, I suppose as a preemptive measure against censorship activists. Made a couple of changes to project to (18, 512) space: https://github.com/oneiroid/stylegan2

[wduo]

@oneiroid Did you run python align_images.py before python encode_images.py?

[rolux]

Karras implemented projecting to latent space right there in the original repo yo. Don't think it can be improved any more. He's using tiled (1, 512) dlatent though, I suppose as a preemptive measure against censorship activists. Made a couple of changes to project to (18, 512) space: https://github.com/oneiroid/stylegan2

@oneiroid: Uh, you're totally right!

Optimization video: https://youtu.be/DqOXWb1fvHk

This looks pretty good to me!

[rolux]

In the meantime, I've done two things: ported this repo to StyleGAN2 (link), at least up to the point where the tutorial notebook runs; and, in my own port of Puzer's encoder (link), added a project_images.py equivalent to encode_images.py.

My impression is that there is a lot of cool stuff in here that would be great to have in StyleGAN2. On the other hand, from a "user" - as opposed to "researcher" - perspective, a two-line change to the projector, plus a bit of glue around it, seems kind of hard to beat.

[woctezuma]

Karras implemented projecting to latent space right there in the original repo yo. Don't think it can be improved any more. He's using tiled (1, 512) dlatent though, I suppose as a preemptive measure against censorship activists. Made a couple of changes to project to (18, 512) space: https://github.com/oneiroid/stylegan2

Hello @oneiroid, would you mind explaining what you meant as "preemptive measure against censorship activists"? I am trying to understand this (18, 512) change (rolux/stylegan2encoder#21), and so far, my understanding is that the visual fidelity is better but the semantic fidelity (for interpolation, etc.) is not. I assume that is the reason why Nvidia's implementation uses (1, 512). I would be curious about your other explanation.

Edit: I have found these related issues (#1, #2), including one created by you.

[oneiroid]

Hi @woctezuma, I positively wouldn't mind explaining :))
Using (1, 512) dlatents you will never get good projection for any face that was not present in the training dataset.
Using (18, 512) dlats you can project any face. And then you can apply facial attributes directions - emotions, age, gender.
Even celebrity faces! And them celebrities got their faces copyrighted and sold for good money.
So there could've been deepfakes-style hysteria and calls to forbid everything starting from GANs down to internet and propagation of light reflected off their precioussssssss privately owned faces.
Karras and Nvidia do not need such sick activity around them. I hope I made myself clear. I'm a bit distressed recently by all the sick things happening, tried my best :)

[woctezuma]

Thank you for clarifying. It helps me in my understanding of this change!