Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for training with Grayscale images? #10677

Open
DavidGill159 opened this issue Jan 28, 2025 · 0 comments
Open

Support for training with Grayscale images? #10677

DavidGill159 opened this issue Jan 28, 2025 · 0 comments

Comments

@DavidGill159
Copy link

DavidGill159 commented Jan 28, 2025

I am trying to train an unconditional diffusion model on grayscale images using your pipeline. When running training with the default parameters I discovered inferred images that contained colour (specifically green). Where it learnt such colours from I do not know but I would predict the issue lies within the initial processing of the image set:

images = [augmentations(image.convert("RGB")) for image in examples["image"]]

as such I created a fork of this repo and changed this line to:

images = [augmentations(image.convert("L")) for image in examples["image"]]

I also updated the model configuration (UNet2DModel) to work with single-channel inputs and outputs by setting in_channels=1 and out_channels=1 when initialising the model.

Am I on the right track? or does the resolution lie elsewhere? I also noticed the resolution of the inferred images is very poor; not on par with the training set. What parameters can I adjust to improve this?
Ultimately I am interested in a diffusion model that focuses more on the textural composition of images, rather than the colour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant