Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support LPIPS when using LAB colorspace. #18

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Support LPIPS when using LAB colorspace. #18

wants to merge 3 commits into from

Conversation

pcuenca
Copy link
Collaborator

@pcuenca pcuenca commented Aug 1, 2022

Addresses #12.

I also double-checked that the input for LPIPS are images in the range [-1, 1], everything looks correct in that regard.

@borisdayma
Copy link
Collaborator

Can you call the function to_rgb (or similar) and make it a noop when the format is already RGB?

@borisdayma
Copy link
Collaborator

borisdayma commented Aug 1, 2022

Also can you check if it affects the prepare_dataset notebook and the wandb logging (may need to use a jax.device_get there).

@pcuenca
Copy link
Collaborator Author

pcuenca commented Aug 1, 2022

Can you call the function to_rgb (or similar) and make it a noop when the format is already RGB?

I didn't want to call it to_rgb because then it looks semantically similar to logits_to_image, but images are returned in [0, 1] and this one works in [-1, 1]. But maybe it's clearer that way.

@borisdayma
Copy link
Collaborator

Can you call the function to_rgb (or similar) and make it a noop when the format is already RGB?

I didn't want to call it to_rgb because then it looks semantically similar to logits_to_image, but images are returned in [0, 1] and this one works in [-1, 1]. But maybe it's clearer that way.

Maybe to_lpips? Pretty close to what you had then.

@pcuenca
Copy link
Collaborator Author

pcuenca commented Aug 1, 2022

Also can you check if it affects the prepare_dataset notebook and the wandb logging (may need to use a jax.device_get there).

Sure, good point. But this was branched from main not stylegan. I'll check if there are any changes needed and if so maybe it makes sense to submit as a PR to stylegan instead.

@pcuenca
Copy link
Collaborator Author

pcuenca commented Aug 2, 2022

I addressed a couple of @borisdayma's comments, but it turns out we can't use tfio.experimental.color.lab_to_rgb in jitted code because there's an implicit conversion to numpy here.

I checked the color conversion functions in dm_pix (https://github.com/deepmind/dm_pix/blob/master/dm_pix/_src/color_conversion.py), but they don't have LAB support yet.

We can port TensorFlow's implementation to JAX, but maybe we have more important things to do first. What do you think @borisdayma, @patil-suraj ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants