Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for torch.float16 to half VRAM requirements #4

Open
thekitchenscientist opened this issue Feb 16, 2023 · 0 comments
Open

Support for torch.float16 to half VRAM requirements #4

thekitchenscientist opened this issue Feb 16, 2023 · 0 comments

Comments

@thekitchenscientist
Copy link

thekitchenscientist commented Feb 16, 2023

If you add the code:
if latents_dtype not in [torch.float, torch.double]: noise_guidance_edit_tmp = noise_guidance_edit_tmp.float()
before the torch.quantile() calculation and then

if latents_dtype not in [torch.float, torch.double]: noise_pred = noise_pred.to(torch.float16)
prior to # compute the previous noisy sample x_t -> x_t-1

the model runs with half the VRAM, which with xformers/attention slicing enabled on the pipe means the model can run with just 5-6GB VRAM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant