ETC-TON:Enhancing Temporal Consistency of video-based virtual Try-ON with latent diffusion models

NOTE: Thanks to ladi-vton, DisCo for the inspiration, and any related discussions are welcome.

Result Visualization

Instructions

• Improved 3D denoising UNet, improved face reconstruction performance and temporal consistency, using VVT dataset, fine-tuning based on stable-diffusion-2-inpainting
• Preprocessing dataset based on Densepose, Human-Parsing. The processed files will be uploaded soon
• Based on a pre-trained self-developed clothing deformation model, images of the warped clothing will also be released. The deformation model can be replaced by the existing graph virtual try-on deformation model

Release Plan

• Test results visualization
• Code for model
• Trained checkpoint
• Code for evaluation on VVT dataset
• Warping model
• Code for training
• Code for general inference

Model checkpoint

• VAE
• UNet
• Vision Encoder Projector