Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add Distillation example for consistency models #2

Open
wants to merge 205 commits into
base: consistency-models-pipeline
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
205 commits
Select commit Hold shift + click to select a range
51843fd
Refactor full determinism (#3485)
patrickvonplaten May 22, 2023
6dd3871
Fix DPM single (#3413)
patrickvonplaten May 22, 2023
194b0a4
Add `use_Karras_sigmas` to DPMSolverSinglestepScheduler (#3476)
Isotr0py May 22, 2023
0160e51
Adds local_files_only bool to prevent forced online connection (#3486)
w4ffl35 May 22, 2023
a2874af
make style
patrickvonplaten May 22, 2023
229fd8c
[Docs] Korean translation (optimization, training) (#3488)
Snailpong May 22, 2023
b8b5daa
DataLoader respecting EXIF data in Training Images (#3465)
Ambrosiussen May 22, 2023
2b56e8c
make style
patrickvonplaten May 22, 2023
f3d570c
feat: allow disk offload for diffuser models (#3285)
hari10599 May 22, 2023
c4359d6
[Community] reference only control (#3435)
okotaku May 22, 2023
64bf5d3
Support for cross-attention bias / mask (#2634)
Birch-san May 22, 2023
67cd460
do not scale the initial global step by gradient accumulation steps w…
williamberman May 22, 2023
2f997f3
Fix bug in panorama pipeline when using dpmsolver scheduler (#3499)
Isotr0py May 23, 2023
edc6505
[Community Pipelines]Accelerate inference of stable diffusion by IPEX…
yingjie-han May 23, 2023
b134f6a
[Community] ControlNet Reference (#3508)
okotaku May 23, 2023
d4197bf
Allow custom pipeline loading (#3504)
patrickvonplaten May 23, 2023
9e2734a
Make sure Diffusers works even if Hub is down (#3447)
patrickvonplaten May 23, 2023
84ce50f
Improve README (#3524)
patrickvonplaten May 23, 2023
b402604
Update README.md (#3525)
patrickvonplaten May 23, 2023
abab61d
Update README.md
patrickvonplaten May 23, 2023
bde2cb5
Run `torch.compile` tests in separate subprocesses (#3503)
pcuenca May 23, 2023
c13dbd5
fix attention mask pad check (#3531)
williamberman May 23, 2023
db56f8a
explicit broadcasts for assignments (#3535)
williamberman May 24, 2023
8e69708
[Examples/DreamBooth] refactor save_model_card utility in dreambooth …
sayakpaul May 24, 2023
a94977b
Fix panorama to support all schedulers (#3546)
Isotr0py May 24, 2023
f19f128
Add open parti prompts to docs (#3549)
patrickvonplaten May 25, 2023
03b7a84
Add Kandinsky 2.1 (#3308)
yiyixuxu May 25, 2023
e5215de
fix broken change for vq pipeline (#3563)
yiyixuxu May 26, 2023
d114d80
[Stable Diffusion Inpainting] Allow standard text-to-img checkpoints …
patrickvonplaten May 26, 2023
d8ce53a
Fix loaded_token reference before definition (#3523)
eminn May 26, 2023
ffa33d6
renamed variable to input_ and output_ (#3507)
vikasmech May 26, 2023
66356e7
Correct inpainting controlnet docs (#3572)
patrickvonplaten May 26, 2023
bf16a97
Fix controlnet guess mode euler (#3571)
patrickvonplaten May 26, 2023
067a9ef
Add initial training script
ayushtues May 26, 2023
7948db8
[docs] Add AttnProcessor to docs (#3474)
stevhliu May 26, 2023
352ca31
[WIP] Add UniDiffuser model and pipeline (#2963)
dg845 May 26, 2023
67cf044
Fix to apply LoRAXFormersAttnProcessor instead of LoRAAttnProcessor w…
takuma104 May 26, 2023
1d1f648
fix dreambooth attention mask (#3541)
linbo0518 May 26, 2023
bdc75e7
[IF super res] correctly normalize PIL input (#3536)
williamberman May 26, 2023
ab98676
[docs] Maintenance (#3552)
stevhliu May 26, 2023
9917c32
[docs] update the broken links (#3568)
brandonJY May 26, 2023
5559d04
[docs] Working with different formats (#3534)
stevhliu May 26, 2023
c6ae883
remove print statements from attention processor. (#3592)
sayakpaul May 29, 2023
c0f867a
Fix temb attention (#3607)
patrickvonplaten May 30, 2023
c059cc0
[docs] update the broken links (#3577)
kadirnar May 30, 2023
0612f48
[UniDiffuser Tests] Fix some tests (#3609)
sayakpaul May 30, 2023
35a7404
#3487 Fix inpainting strength for various samplers (#3532)
rupertmenneer May 30, 2023
6cbddf5
[Community] Support StableDiffusionTilingPipeline (#3586)
kadirnar May 30, 2023
07ef485
[Community, Enhancement] Add reference tricks in README (#3589)
okotaku May 30, 2023
799f5b4
[Feat] Enable State Dict For Textual Inversion Loader (#3439)
ghunkins May 30, 2023
bb22d54
[Community] CLIP Guided Images Mixing with Stable DIffusion Pipeline …
TheDenk May 30, 2023
160c377
Make style
patrickvonplaten May 30, 2023
a359ab4
Update README.md
patrickvonplaten May 30, 2023
9d3ff07
fix tests (#3614)
patrickvonplaten May 30, 2023
0cc3a7a
Make sure we also change the config when setting `encoder_hid_dim_typ…
patrickvonplaten May 30, 2023
7d0ac4e
goodbye frog (#3617)
williamberman May 30, 2023
abb89da
update code to reflect latest changes as of May 30th (#3616)
prathikr May 31, 2023
f751b88
update dreambooth lora to work with IF stage II (#3560)
williamberman May 31, 2023
4f14b36
Full Dreambooth IF stage II upscaling (#3561)
williamberman May 31, 2023
55dbfa0
[Docs] include the instruction-tuning blog link in the InstructPix2Pi…
sayakpaul Jun 2, 2023
32ea214
[Kandinsky] Improve kandinsky API a bit (#3636)
patrickvonplaten Jun 2, 2023
8e552bb
Support Kohya-ss style LoRA file format (in a limited capacity) (#3437)
takuma104 Jun 2, 2023
a6c7b5b
Iterate over unique tokens to avoid duplicate replacements for multiv…
lachlan-nicholson Jun 2, 2023
f1d4743
fixed typo in example train_text_to_image.py (#3608)
kashif Jun 2, 2023
0e86881
fix inpainting pipeline when providing initial latents (#3641)
yiyixuxu Jun 2, 2023
0dbdc0c
[Community Doc] Updated the filename and readme file. (#3634)
kadirnar Jun 2, 2023
d3717e6
add Stable Diffusion TensorRT Inpainting pipeline (#3642)
asfiyab-nvidia Jun 2, 2023
b7af946
set config from original module but set compiled module on class (#3650)
williamberman Jun 2, 2023
5911a3a
dreambooth if docs - stage II, more info (#3628)
williamberman Jun 2, 2023
7a39691
linting fix (#3653)
williamberman Jun 2, 2023
b95cbdf
Set step_rules correctly for piecewise_constant scheduler (#3605)
0x1355 Jun 5, 2023
de45af4
Allow setting num_cycles for cosine_with_restarts lr scheduler (#3606)
0x1355 Jun 5, 2023
523a50a
[docs] Load A1111 LoRA (#3629)
stevhliu Jun 5, 2023
0fc2fb7
dreambooth upscaling fix added latents (#3659)
williamberman Jun 5, 2023
262d539
Correct multi gpu dreambooth (#3673)
patrickvonplaten Jun 5, 2023
1994dbc
Fix from_ckpt not working properly on windows (#3666)
LyubimovVladislav Jun 5, 2023
a15c168
Merge branch 'consistency-models-pipeline' into consistency-models-ex…
ayushtues Jun 5, 2023
d0416ab
Update Compel documentation for textual inversions (#3663)
pdoane Jun 5, 2023
995bbcb
[UniDiffuser test] fix one test so that it runs correctly on V100 (#3…
sayakpaul Jun 5, 2023
1a6a647
[docs] More API fixes (#3640)
stevhliu Jun 5, 2023
5990014
[WIP]Vae preprocessor refactor (PR1) (#3557)
yiyixuxu Jun 5, 2023
462956b
small tweaks for parsing thibaudz controlnet checkpoints (#3657)
williamberman Jun 5, 2023
41ae670
move activation dispatches into helper function (#3656)
williamberman Jun 5, 2023
a8b0f42
[docs] Fix link to loader method (#3680)
stevhliu Jun 6, 2023
b45204e
Add function to remove monkey-patch for text encoder LoRA (#3649)
takuma104 Jun 6, 2023
8669e83
[LoRA] feat: add lora attention processor for pt 2.0. (#3594)
sayakpaul Jun 6, 2023
017ee16
refactor Image processor for x4 upscaler (#3692)
yiyixuxu Jun 6, 2023
de16f64
feat: when using PT 2.0 use LoRAAttnProcessor2_0 for text enc LoRA. (…
sayakpaul Jun 6, 2023
10f4ecd
Fix the Kandinsky docstring examples (#3695)
freespirit Jun 6, 2023
11b3002
Support views batch for panorama (#3632)
Isotr0py Jun 6, 2023
2de9e2d
Fix from_ckpt for Stable Diffusion 2.x (#3662)
ctrysbita Jun 6, 2023
74fd735
Add draft for lora text encoder scale (#3626)
patrickvonplaten Jun 6, 2023
12a232e
Fix schedulers zero SNR and rescale classifier free guidance (#3664)
Max-We Jun 7, 2023
fdec231
[Tests] Run slow matrix sequentially (#3500)
pcuenca Jun 7, 2023
cd9d091
Fixes eval generator init in `train_text_to_image_lora.py` (#3678)
vvvm23 Jun 7, 2023
803d653
Fix custom releases (#3708)
patrickvonplaten Jun 7, 2023
cd61869
[Community] Support StableDiffusionCanvasPipeline (#3590)
kadirnar Jun 7, 2023
8caa530
[doc build] Use secrets (#3707)
Jun 7, 2023
500a3ff
[docs] add image processor documentation (#3710)
yiyixuxu Jun 7, 2023
a06317a
[Actions] Fix actions (#3712)
patrickvonplaten Jun 7, 2023
79fa94e
Apply deprecations from Accelerate (#3714)
muellerzr Jun 8, 2023
f523b11
Fix loading if unexpected keys are present (#3720)
patrickvonplaten Jun 8, 2023
c42f6ee
Post 0.17.0 release (#3721)
patrickvonplaten Jun 8, 2023
787e65a
Minor
ayushtues Jun 9, 2023
50136d0
Merge branch 'consistency-models-pipeline' into consistency-models-ex…
ayushtues Jun 9, 2023
6530b17
Add training code
ayushtues Jun 9, 2023
8f858cb
Fix bugs in training
ayushtues Jun 9, 2023
e56b870
Remove some args, add target model
ayushtues Jun 9, 2023
0536196
remove seed (#3734)
yiyixuxu Jun 9, 2023
27af55d
build docs
patrickvonplaten Jun 10, 2023
e891b00
build docs
patrickvonplaten Jun 10, 2023
1d50f47
Merge branch 'main' of https://github.com/huggingface/diffusers
patrickvonplaten Jun 10, 2023
abbfe4b
fix zh
patrickvonplaten Jun 10, 2023
11aa105
Correct Token to upload docs (#3744)
patrickvonplaten Jun 12, 2023
790212f
Correct another push token (#3745)
patrickvonplaten Jun 12, 2023
38adcd2
[Stable Diffusion Inpaint & ControlNet inpaint] Correct timestep inpa…
patrickvonplaten Jun 12, 2023
1488180
attention weight loading fix
ayushtues Jun 12, 2023
b2b13cd
[Documentation] Replace dead link to Flax install guide (#3739)
JeLuF Jun 12, 2023
f46b22b
[documentation] grammatical fixes in installation.mdx (#3735)
LiamSwayne Jun 12, 2023
a812fb6
Text2video zero refinements (#3733)
19and99 Jun 12, 2023
ef95907
[Tests] Relax tolerance of flaky failing test (#3755)
patrickvonplaten Jun 12, 2023
34d14d7
[MultiControlNet] Allow save and load (#3747)
patrickvonplaten Jun 12, 2023
ce55049
Update pipeline_flax_stable_diffusion_controlnet.py (#3306)
jfozard Jun 13, 2023
7761b89
update conversion script for Kandinsky unet (#3766)
yiyixuxu Jun 14, 2023
f96b760
[docs] Fix Colab notebook cells (#3777)
stevhliu Jun 14, 2023
027a365
[Bug Report template] modify the issue template to include core maint…
sayakpaul Jun 15, 2023
98e1381
Renaming ema model to target
ayushtues Jun 15, 2023
baefc87
Add some comments
ayushtues Jun 15, 2023
1ae15fa
[Enhance] Update reference (#3723)
okotaku Jun 15, 2023
2715079
Fix broken cpu-offloading in legacy inpainting SD pipeline (#3773)
cmdr2 Jun 15, 2023
908e5e9
Fix some bad comment in training scripts (#3798)
patrickvonplaten Jun 15, 2023
75124fc
Added LoRA loading to `StableDiffusionKDiffusionPipeline` (#3751)
tripathiarpan20 Jun 15, 2023
231bdf2
UnCLIP Image Interpolation -> Keep same initial noise across interpol…
Abhinay1997 Jun 15, 2023
77f9137
feat: add PR template. (#3786)
sayakpaul Jun 15, 2023
958d9ec
Ldm3d first PR (#3668)
estelleafl Jun 15, 2023
ea8ae8c
Complete set_attn_processor for prior and vae (#3796)
patrickvonplaten Jun 15, 2023
7bfd237
fix typo (#3800)
Isotr0py Jun 15, 2023
d49e2dd
manual check for checkpoints_total_limit instead of using accelerate …
williamberman Jun 15, 2023
3ddc2b7
[train text to image] add note to loading from checkpoint (#3806)
williamberman Jun 16, 2023
59aefe9
device map legacy attention block weight conversion (#3804)
williamberman Jun 16, 2023
f7cc9ad
[docs] Zero SNR (#3776)
stevhliu Jun 16, 2023
cd460ca
Remove xformers, refactor ckpt resuming
ayushtues Jun 17, 2023
9bc3167
Merge branch 'consistency-models-pipeline' into consistency-models-ex…
ayushtues Jun 18, 2023
9a81642
Add input scaling, disable gradients
ayushtues Jun 19, 2023
6667433
[ldm3d] Fixed small typo (#3820)
estelleafl Jun 19, 2023
4870626
[Examples] Improve the model card pushed from the `train_text_to_imag…
sayakpaul Jun 20, 2023
88eb044
[Docs] add missing pipelines from the overview pages and minor fixes …
sayakpaul Jun 20, 2023
73b125d
[Pipeline] Add new pipeline for ParaDiGMS -- parallel sampling of dif…
AndyShih12 Jun 20, 2023
3ebbaf7
Update control_brightness.mdx (#3825)
dqueue Jun 20, 2023
ef3844d
Support ControlNet models with different number of channels in contro…
JCBrouwer Jun 21, 2023
95ea538
Add ddpm kandinsky (#3783)
yiyixuxu Jun 21, 2023
1f02087
[docs] More API stuff (#3835)
stevhliu Jun 21, 2023
0bab447
relax tol attention conversion test (#3842)
williamberman Jun 21, 2023
13e781f
fix: random module seeding (#3846)
sayakpaul Jun 22, 2023
0c6d1bc
fix audio_diffusion tests (#3850)
teticio Jun 22, 2023
88d2694
Correct bad attn naming (#3797)
patrickvonplaten Jun 22, 2023
5df2acf
[Conversion] Small fixes (#3848)
patrickvonplaten Jun 22, 2023
5e3f8ff
Fix some audio tests (#3841)
patrickvonplaten Jun 22, 2023
fc6acb6
[Docs] add: contributor note in the paradigms docs. (#3852)
sayakpaul Jun 22, 2023
61916fe
Update Habana Gaudi doc (#3863)
regisss Jun 24, 2023
9a45d7f
Add guidance start/stop (#3770)
holwech Jun 26, 2023
0bf6aeb
feat: rename single-letter vars in `resnet.py` (#3868)
SauravMaheshkar Jun 28, 2023
35bac5e
Fixing the global_step key not found (#3844)
VincentNeemie Jun 28, 2023
219636f
improve tolerance
patrickvonplaten Jun 28, 2023
1500130
Support for manual CLIP loading in StableDiffusionPipeline - txt2img.…
WadRex Jun 28, 2023
c7469eb
fix sde add noise typo (#3839)
UranusITS Jun 28, 2023
49949f3
[Tests] add test for checking soft dependencies. (#3847)
sayakpaul Jun 28, 2023
0240407
Merge branch 'consistency-models-pipeline' into consistency-models-ex…
ayushtues Jun 29, 2023
cdf2ae8
[Enhance] Add LoRA rank args in train_text_to_image_lora (#3866)
okotaku Jun 29, 2023
174dcd6
[docs] Model API (#3562)
stevhliu Jun 30, 2023
5439e91
fix/docs: Fix the broken doc links (#3897)
Aisuko Jul 1, 2023
6282506
Add video img2img (#3900)
patrickvonplaten Jul 2, 2023
f911287
fix/doc-code: Updating to the latest version parameters (#3924)
Aisuko Jul 3, 2023
b298484
fix/doc: no import torch issue (#3923)
Aisuko Jul 3, 2023
2e8668f
Correct controlnet out of list error (#3928)
patrickvonplaten Jul 3, 2023
572d8e2
Adding better way to define multiple concepts and also validation cap…
mauricio-repetto Jul 3, 2023
b8a5dda
[ldm3d] Update code to be functional with the new checkpoints (#3875)
estelleafl Jul 3, 2023
332d2bb
Improve memory text to video (#3930)
patrickvonplaten Jul 3, 2023
4e89856
revert automatic chunking (#3934)
patrickvonplaten Jul 3, 2023
1997614
avoid upcasting by assigning dtype to noise tensor (#3713)
prathikr Jul 4, 2023
2837d49
Fix failing np tests (#3942)
patrickvonplaten Jul 4, 2023
07c9a08
Add `timestep_spacing` and `steps_offset` to schedulers (#3947)
pcuenca Jul 5, 2023
aed7499
Add Consistency Models Pipeline (#3492)
dg845 Jul 5, 2023
41ea88f
Update consistency_models.mdx (#3961)
sayakpaul Jul 6, 2023
de14261
Make `UNet2DConditionOutput` pickle-able (#3857)
prathikr Jul 6, 2023
46af982
[Consistency Models] correct checkpoint url in the doc (#3962)
sayakpaul Jul 6, 2023
b62d9a1
[Text-to-video] Add `torch.compile()` compatibility (#3949)
sayakpaul Jul 6, 2023
bc9a8ce
[SD-XL] Add new pipelines (#3859)
patrickvonplaten Jul 6, 2023
7462156
Kandinsky_v22_yiyi (#3936)
yiyixuxu Jul 6, 2023
45f6d52
Add Shap-E (#3742)
yiyixuxu Jul 6, 2023
8bf80fc
disable num attenion heads (#3969)
patrickvonplaten Jul 6, 2023
187ea53
Improve SD XL (#3968)
patrickvonplaten Jul 6, 2023
b8f089c
fix/doc-code: import torch and fix the broken document address (#3941)
Aisuko Jul 6, 2023
38e563d
Fix SD XL Docs (#3971)
patrickvonplaten Jul 6, 2023
51593da
fix main docs
patrickvonplaten Jul 6, 2023
1fbcc78
typo in safetensors (safetenstors) (#3976)
YoraiLevi Jul 7, 2023
8d8b431
Fix code snippet for Audio Diffusion (#3987)
osanseviero Jul 7, 2023
03d829d
feat: add `Dropout` to Flax UNet (#3894)
SauravMaheshkar Jul 7, 2023
88356e6
Merge branch 'main_hf' into consistency-models-examples
ayushtues Jul 7, 2023
ea7d75f
Update training script to main, fix timesteps
ayushtues Jul 10, 2023
a32b869
Fix bug in timestep ordering
ayushtues Jul 10, 2023
8742e4e
Add review suggestions
ayushtues Jul 11, 2023
943c88b
Integrate accelerator better, change model upload
ayushtues Jul 12, 2023
6b58d81
Fix checkpointing and add test
ayushtues Jul 13, 2023
180b3e8
Remove hardcoded configs, add DiffusionPipeline
ayushtues Jul 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions examples/consistency_models/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
accelerate>=0.16.0
torchvision
datasets
3 changes: 3 additions & 0 deletions examples/consistency_models/script.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

accelerate launch train_consistency_distillation.py --dataset_name="cifar10" --resolution=32 --center_crop --random_flip --output_dir="cifar10-32" --train_batch_size=16 --num_epochs=100 --gradient_accumulation_steps=1 --learning_rate=1e-4 --lr_warmup_steps=500 --mixed_precision=no --push_to_hub
Loading