[WIP] Add Distillation example for consistency models - #2

Open

ayushtues wants to merge 205 commits into

dg845:consistency-models-pipelinefrom

ayushtues:consistency-models-examples

Commits on May 22, 2023

Refactor full determinism (#3485)
patrickvonplaten
authored
51843fd
Fix DPM single (#3413)

patrickvonplaten
and
StAlKeR7779
authored
6dd3871
Add `use_Karras_sigmas` to DPMSolverSinglestepScheduler (#3476)
Isotr0py
authored
194b0a4
Adds local_files_only bool to prevent forced online connection (#3486)
w4ffl35
authored
0160e51
make style
patrickvonplaten
committed
a2874af
[Docs] Korean translation (optimization, training) (#3488)

tjdtnsu
and
seriousran
authored
229fd8c
DataLoader respecting EXIF data in Training Images (#3465)
Ambrosiussen
authored
b8b5daa
make style
patrickvonplaten
committed
2b56e8c
feat: allow disk offload for diffuser models (#3285)

authored
f3d570c
[Community] reference only control (#3435)
okotaku
authored
c4359d6
Support for cross-attention bias / mask (#2634)
Birch-san
authored
64bf5d3
do not scale the initial global step by gradient accumulation steps when loading from checkpoint (#3506)
williamberman
authored
67cd460

Commits on May 23, 2023

Fix bug in panorama pipeline when using dpmsolver scheduler (#3499)
Isotr0py
authored
2f997f3
[Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU (#3105)

yingjie-han
and
pcuenca
authored
edc6505
[Community] ControlNet Reference (#3508)

okotaku
and
patrickvonplaten
authored
b134f6a
Allow custom pipeline loading (#3504)
patrickvonplaten
authored
d4197bf
Make sure Diffusers works even if Hub is down (#3447)
patrickvonplaten
authored
9e2734a
Improve README (#3524)
patrickvonplaten
authored
84ce50f
Update README.md (#3525)
patrickvonplaten
authored
b402604
Update README.md
patrickvonplaten
authored
abab61d
Run `torch.compile` tests in separate subprocesses (#3503)
pcuenca
authored
bde2cb5
fix attention mask pad check (#3531)
williamberman
authored
c13dbd5

Commits on May 24, 2023

Commits on May 25, 2023

Add open parti prompts to docs (#3549)
patrickvonplaten
authored
f19f128
Add Kandinsky 2.1 (#3308)

authored
03b7a84

Commits on May 26, 2023

fix broken change for vq pipeline (#3563)

yiyixuxu
and
yiyixuxu
authored
e5215de
[Stable Diffusion Inpainting] Allow standard text-to-img checkpoints to be useable for SD inpainting (#3533)
patrickvonplaten
authored
d114d80
Fix loaded_token reference before definition (#3523)
eminn
authored
d8ce53a
renamed variable to input_ and output_ (#3507)
vikasmech
authored
ffa33d6
Correct inpainting controlnet docs (#3572)
patrickvonplaten
authored
66356e7
Fix controlnet guess mode euler (#3571)

patrickvonplaten
and
offchan42
authored
bf16a97
Add initial training script
ayushtues
committed
067a9ef
[docs] Add AttnProcessor to docs (#3474)
stevhliu
authored
7948db8
[WIP] Add UniDiffuser model and pipeline (#2963)

authored
352ca31
Fix to apply LoRAXFormersAttnProcessor instead of LoRAAttnProcessor when xFormers is enabled (#3556)
takuma104
authored
67cf044
fix dreambooth attention mask (#3541)
linbo0518
authored
1d1f648
[IF super res] correctly normalize PIL input (#3536)
williamberman
authored
bdc75e7
[docs] Maintenance (#3552)
stevhliu
authored
ab98676
[docs] update the broken links (#3568)
brandonJY
authored
9917c32
[docs] Working with different formats (#3534)
stevhliu
authored
5559d04

Commits on May 29, 2023

remove print statements from attention processor. (#3592)
sayakpaul
authored
c6ae883

Commits on May 30, 2023

Fix temb attention (#3607)
patrickvonplaten
authored
c0f867a
[docs] update the broken links (#3577)
kadirnar
authored
c059cc0
[UniDiffuser Tests] Fix some tests (#3609)
sayakpaul
authored
0612f48
#3487 Fix inpainting strength for various samplers (#3532)

rupertmenneer
and
patrickvonplaten
authored
35a7404
[Community] Support StableDiffusionTilingPipeline (#3586)
kadirnar
authored
6cbddf5
[Community, Enhancement] Add reference tricks in README (#3589)
okotaku
authored
07ef485
[Feat] Enable State Dict For Textual Inversion Loader (#3439)

ghunkins
and
patrickvonplaten
authored
799f5b4
[Community] CLIP Guided Images Mixing with Stable DIffusion Pipeline (#3587)

TheDenk
and
patrickvonplaten
authored
bb22d54
Make style
patrickvonplaten
committed
160c377
Update README.md
patrickvonplaten
authored
a359ab4
fix tests (#3614)
patrickvonplaten
authored
9d3ff07
Make sure we also change the config when setting `encoder_hid_dim_type=="text_proj"` and allow xformers (#3615)
patrickvonplaten
authored
0cc3a7a
goodbye frog (#3617)
williamberman
authored
7d0ac4e

Commits on May 31, 2023

Commits on Jun 2, 2023

[Docs] include the instruction-tuning blog link in the InstructPix2Pix docs (#3644)
sayakpaul
authored
55dbfa0
[Kandinsky] Improve kandinsky API a bit (#3636)

patrickvonplaten
and
yiyixuxu
authored
32ea214
Support Kohya-ss style LoRA file format (in a limited capacity) (#3437)

takuma104
and
sayakpaul
authored
8e552bb
Iterate over unique tokens to avoid duplicate replacements for multivector embeddings (#3588)
lachlan-nicholson
authored
a6c7b5b
fixed typo in example train_text_to_image.py (#3608)
kashif
authored
f1d4743
fix inpainting pipeline when providing initial latents (#3641)

yiyixuxu
and
yiyixuxu
authored
0e86881
[Community Doc] Updated the filename and readme file. (#3634)
kadirnar
authored
0dbdc0c
add Stable Diffusion TensorRT Inpainting pipeline (#3642)

asfiyab-nvidia
and
patrickvonplaten
authored
d3717e6
set config from original module but set compiled module on class (#3650)
williamberman
authored
b7af946
dreambooth if docs - stage II, more info (#3628)

authored
5911a3a
linting fix (#3653)
williamberman
authored
7a39691

Commits on Jun 5, 2023

Commits on Jun 6, 2023

[docs] Fix link to loader method (#3680)
stevhliu
authored
a8b0f42
Add function to remove monkey-patch for text encoder LoRA (#3649)
takuma104
authored
b45204e
[LoRA] feat: add lora attention processor for pt 2.0. (#3594)
sayakpaul
authored
8669e83
refactor Image processor for x4 upscaler (#3692)

yiyixuxu
and
yiyixuxu
authored
017ee16
feat: when using PT 2.0 use LoRAAttnProcessor2_0 for text enc LoRA. (#3691)
sayakpaul
authored
de16f64
Fix the Kandinsky docstring examples (#3695)
freespirit
authored
10f4ecd
Support views batch for panorama (#3632)

Isotr0py
and
sayakpaul
authored
11b3002
Fix from_ckpt for Stable Diffusion 2.x (#3662)
ctrysbita
authored
2de9e2d
Add draft for lora text encoder scale (#3626)

patrickvonplaten
and
sayakpaul
authored
74fd735

Commits on Jun 7, 2023

Fix schedulers zero SNR and rescale classifier free guidance (#3664)

authored
12a232e
[Tests] Run slow matrix sequentially (#3500)
pcuenca
authored
fdec231
Fixes eval generator init in `train_text_to_image_lora.py` (#3678)
vvvm23
authored
cd9d091
Fix custom releases (#3708)
patrickvonplaten
authored
803d653
[Community] Support StableDiffusionCanvasPipeline (#3590)
kadirnar
authored
cd61869
[doc build] Use secrets (#3707)

Mishig
and
patrickvonplaten
authored
8caa530
[docs] add image processor documentation (#3710)

yiyixuxu
and
yiyixuxu
authored
500a3ff
[Actions] Fix actions (#3712)
patrickvonplaten
authored
a06317a

Commits on Jun 8, 2023

Apply deprecations from Accelerate (#3714)
muellerzr
authored
79fa94e
Fix loading if unexpected keys are present (#3720)
patrickvonplaten
authored
f523b11
Post 0.17.0 release (#3721)
patrickvonplaten
authored
c42f6ee

Commits on Jun 9, 2023

Minor
ayushtues
committed
787e65a
Merge branch 'consistency-models-pipeline' into consistency-models-examples
ayushtues
committed
50136d0
Add training code
ayushtues
committed
6530b17
Fix bugs in training
ayushtues
committed
8f858cb
Remove some args, add target model
ayushtues
committed
e56b870
remove seed (#3734)

yiyixuxu
and
yiyixuxu
authored
0536196

Commits on Jun 10, 2023

build docs
patrickvonplaten
committed
27af55d
build docs
patrickvonplaten
committed
e891b00
Merge branch 'main' of https://github.com/huggingface/diffusers
patrickvonplaten
committed
1d50f47
fix zh
patrickvonplaten
committed
abbfe4b

Commits on Jun 12, 2023

Correct Token to upload docs (#3744)
patrickvonplaten
authored
11aa105
Correct another push token (#3745)
patrickvonplaten
authored
790212f
[Stable Diffusion Inpaint & ControlNet inpaint] Correct timestep inpaint (#3749)
patrickvonplaten
authored
38adcd2
attention weight loading fix
ayushtues
committed
1488180
[Documentation] Replace dead link to Flax install guide (#3739)
JeLuF
authored
b2b13cd
[documentation] grammatical fixes in installation.mdx (#3735)
LiamSwayne
authored
f46b22b
Text2video zero refinements (#3733)

19and99
and
sayakpaul
authored
a812fb6
[Tests] Relax tolerance of flaky failing test (#3755)
patrickvonplaten
authored
ef95907
[MultiControlNet] Allow save and load (#3747)
patrickvonplaten
authored
34d14d7

Commits on Jun 13, 2023

Update pipeline_flax_stable_diffusion_controlnet.py (#3306)
jfozard
authored
ce55049

Commits on Jun 14, 2023

update conversion script for Kandinsky unet (#3766)

yiyixuxu
and
yiyixuxu
authored
7761b89
[docs] Fix Colab notebook cells (#3777)
stevhliu
authored
f96b760

Commits on Jun 15, 2023

[Bug Report template] modify the issue template to include core maintainers. (#3785)

sayakpaul
and
patrickvonplaten
authored
027a365
Renaming ema model to target
ayushtues
committed
98e1381
Add some comments
ayushtues
committed
baefc87
[Enhance] Update reference (#3723)

okotaku
and
patrickvonplaten
authored
1ae15fa
Fix broken cpu-offloading in legacy inpainting SD pipeline (#3773)
cmdr2
authored
2715079
Fix some bad comment in training scripts (#3798)
patrickvonplaten
authored
908e5e9
Added LoRA loading to `StableDiffusionKDiffusionPipeline` (#3751)
tripathiarpan20
authored
75124fc
UnCLIP Image Interpolation -> Keep same initial noise across interpolation steps (#3782)
Abhinay1997
authored
231bdf2
feat: add PR template. (#3786)

sayakpaul
and
patrickvonplaten
authored
77f9137
Ldm3d first PR (#3668)

authored
958d9ec
Complete set_attn_processor for prior and vae (#3796)

patrickvonplaten
and
sayakpaul
authored
ea8ae8c
fix typo (#3800)
Isotr0py
authored
7bfd237
manual check for checkpoints_total_limit instead of using accelerate (#3681)
williamberman
authored
d49e2dd

Commits on Jun 16, 2023

Commits on Jun 17, 2023

Remove xformers, refactor ckpt resuming
ayushtues
committed
cd460ca

Commits on Jun 18, 2023

Merge branch 'consistency-models-pipeline' into consistency-models-examples
ayushtues
committed
9bc3167

Commits on Jun 19, 2023

Add input scaling, disable gradients
ayushtues
committed
9a81642
[ldm3d] Fixed small typo (#3820)

estelleafl
and
Aflalo
authored
6667433

Commits on Jun 20, 2023

Commits on Jun 21, 2023

Support ControlNet models with different number of channels in control images (#3815)
JCBrouwer
authored
ef3844d
Add ddpm kandinsky (#3783)

yiyixuxu
and
yiyixuxu
authored
95ea538
[docs] More API stuff (#3835)
stevhliu
authored
1f02087
relax tol attention conversion test (#3842)
williamberman
authored
0bab447

Commits on Jun 22, 2023

fix: random module seeding (#3846)
sayakpaul
authored
13e781f
fix audio_diffusion tests (#3850)
teticio
authored
0c6d1bc
Correct bad attn naming (#3797)

patrickvonplaten
and
pcuenca
authored
88d2694
[Conversion] Small fixes (#3848)
patrickvonplaten
authored
5df2acf
Fix some audio tests (#3841)
patrickvonplaten
authored
5e3f8ff
[Docs] add: contributor note in the paradigms docs. (#3852)
sayakpaul
authored
fc6acb6

Commits on Jun 24, 2023

Update Habana Gaudi doc (#3863)
regisss
authored
61916fe

Commits on Jun 26, 2023

Add guidance start/stop (#3770)

holwech
and
patrickvonplaten
authored
9a45d7f

Commits on Jun 28, 2023

feat: rename single-letter vars in `resnet.py` (#3868)
SauravMaheshkar
authored
0bf6aeb
Fixing the global_step key not found (#3844)

VincentNeemie
and
patrickvonplaten
authored
35bac5e
improve tolerance
patrickvonplaten
committed
219636f
Support for manual CLIP loading in StableDiffusionPipeline - txt2img. (#3832)

WadRex
and
patrickvonplaten
authored
1500130
fix sde add noise typo (#3839)
UranusITS
authored
c7469eb
[Tests] add test for checking soft dependencies. (#3847)
sayakpaul
authored
49949f3

Commits on Jun 29, 2023

Commits on Jun 30, 2023

[docs] Model API (#3562)
stevhliu
authored
174dcd6

Commits on Jul 1, 2023

fix/docs: Fix the broken doc links (#3897)

Aisuko
and
pcuenca
authored
5439e91

Commits on Jul 2, 2023

Add video img2img (#3900)
patrickvonplaten
authored
6282506

Commits on Jul 3, 2023

fix/doc-code: Updating to the latest version parameters (#3924)
Aisuko
authored
f911287
fix/doc: no import torch issue (#3923)
Aisuko
authored
b298484
Correct controlnet out of list error (#3928)
patrickvonplaten
authored
2e8668f
Adding better way to define multiple concepts and also validation capabilities. (#3807)

mauricio-repetto
and
patrickvonplaten
authored
572d8e2
[ldm3d] Update code to be functional with the new checkpoints (#3875)

authored
b8a5dda
Improve memory text to video (#3930)

patrickvonplaten
and
pcuenca
authored
332d2bb
revert automatic chunking (#3934)
patrickvonplaten
authored
4e89856

Commits on Jul 4, 2023

avoid upcasting by assigning dtype to noise tensor (#3713)

authored
1997614
Fix failing np tests (#3942)
patrickvonplaten
authored
2837d49

Commits on Jul 5, 2023

Add `timestep_spacing` and `steps_offset` to schedulers (#3947)

pcuenca
and
patrickvonplaten
authored
07c9a08
Add Consistency Models Pipeline (#3492)

authored
aed7499

Commits on Jul 6, 2023

Update consistency_models.mdx (#3961)
sayakpaul
authored
41ea88f
Make `UNet2DConditionOutput` pickle-able (#3857)

authored
de14261
[Consistency Models] correct checkpoint url in the doc (#3962)
sayakpaul
authored
46af982
[Text-to-video] Add `torch.compile()` compatibility (#3949)
sayakpaul
authored
b62d9a1
[SD-XL] Add new pipelines (#3859)

authored
bc9a8ce
Kandinsky_v22_yiyi (#3936)

authored
7462156
Add Shap-E (#3742)

authored
45f6d52
disable num attenion heads (#3969)
patrickvonplaten
authored
8bf80fc
Improve SD XL (#3968)
patrickvonplaten
authored
187ea53
fix/doc-code: import torch and fix the broken document address (#3941)
Aisuko
authored
b8f089c
Fix SD XL Docs (#3971)
patrickvonplaten
authored
38e563d
fix main docs
patrickvonplaten
committed
51593da

Commits on Jul 7, 2023

Commits on Jul 10, 2023

Update training script to main, fix timesteps
ayushtues
committed
ea7d75f
Fix bug in timestep ordering
ayushtues
committed
a32b869

Commits on Jul 11, 2023

Add review suggestions
ayushtues
committed
8742e4e

Commits on Jul 12, 2023

Integrate accelerator better, change model upload
ayushtues
committed
943c88b

Commits on Jul 13, 2023

Fix checkpointing and add test
ayushtues
committed
6b58d81

Commits on Jul 19, 2023

Remove hardcoded configs, add DiffusionPipeline
ayushtues
committed
180b3e8