Skip to content

ENFUGUE Web UI v0.2.3
Compare
Choose a tag to compare
@painebenjamin painebenjamin released this 11 Sep 02:11
· 460 commits to main since this release
0.2.3
0aa936d
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

New Features

1. Image Prompt Adapter


Image Prompting is another tool in your toolbox for using an image to create something new. Here we see the girl with the pearl earring effortlessly being transformed into something reminiscent of, but distinct from, Vermeer's classic.

Tencent's AI Lab has released Image Prompt (IP) Adapter, a new method for controlling Stable Diffusion with an input image that provides a huge amount of flexibility, with more consistency than standard image-based inference, and more freedom than than ControlNet images. The best part about it - it works alongside all other control techniques, giving us dozens of new combinations of control methods users can employ.

2. DWPose*, Pose ControlNet XL

*DWPose is currently only available for users managing their own environments. Portable and docker users can still use OpenPose as before.


ControlNet Pose XL is used to match human poses when using SDXL models. Source image by Loris Boulingez via Unsplash.

IDEA Research has released DWPose, a new AI model for detecting human poses, including fingers and faces, faster and more accurately than ever before.

In addition, a community member named Thibaud Zamora has released OpenPose ControlNet for SDXL, which is now the third SDXL ControlNet after Canny Edge and Depth.

You only need to select ControlNet pose to use it. In order to use DWPose, users managing their own environments must execute the following:

#!/usr/bin/env sh
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.1"
mim install "mmdet>=3.1.0"
mim install "mmpose>=1.1.0"

3. Multi ControlNet, Merged Image Nodes


Combine this feature with the new Image Prompt adapter for easy style transfers without needing to prompt anything. The leaf image is by Daria Drewnoska and the dark abstract image by Pawel Czerwinski.

You can now merge images together on the canvas, giving each one it's own assignment(s) toward the overall diffusion plan. Click and drag one image onto another to merge them. You'll be presented the option to drop the image when you bring their headers together (i.e. bring the top of the dragged image to the top of the target image.)

  • The number of images you can use this way is unlimited, however there are some roles that can only be fulfilled by one image.
  • For example, you can only initialize inference from one image (i.e. check 'Use for Inference',) and only use one image for image prompting.
  • ControlNet's can be mixed, matched and reused as desired. Be mindful that each new kind of ControlNet you add increases VRAM requirements significantly. Adding a different image for the same ControlNet only increases VRAM requirements marginally.

4. Multi-Diffusion Options & Speed Boost


The new options for chunking.

Multi-diffusion speed has been improved by as much as 5 iterations per second, thanks to better algorithms for merging chunks. With this comes new options for how these chunks are masked onto each other, blending edges together. The options available as constant, bilinear and gaussian, with the default being bilinear. These images were all generated in 40 steps with a chunking size of 64.


Constant masking can produce sharp details, but visible seams unless you use very low chunking sizes.


Bilinear produces a good mix of edge blending without much detail loss.


Gaussian masking can greatly alter an image, changing where detail is applied, without visible seams.

5. SDXL Textual Inversion Support

With the rising popularity of UnaestheticXL, a negative textual inversion for SDXL by Aikimi, an implementation has been added to Enfugue for loading SDXL TI's. Add them just as you would add other Textual Inversion.

These are a little slow to load at the moment, as this is a temporary workaround pending official implementation into Diffusers.

6. Better Refining


The new refining default options.

Better options have been provided for the refining method. Use the slider at the top to control the step at which the configured refiner takes over denoising, providing a better end result than executing refining as a distinct step.

7. Better Upscaling


The new upscaling interface allows for any number of steps with any scale and method.

Upscaling has been made much more flexible by permitting you to select any number of steps with any set of configuration, rather than only permitting you one upscaling step.

The upscaling amount has additionally been unconstrained, allowing you to use an upscaling algorithm to modify the dimensions of an image by anywhere between 0.5× and 16×.

Full Changelog: 0.2.2...0.2.3

How-To Guide

Installing and Running: Portable Distributions

Select a portable distribution if you'd like to avoid having to install other programs, or want to have an isolated executable file that doesn't interfere with other environments on your system.

Summary

Platform Graphics API File(s) CUDA Version Torch Version
MacOS MPS enfugue-server-0.2.3-macos-ventura-x86_64.tar.gz N/A 2.2.0.dev20230910
Windows CUDA enfugue-server-0.2.3-win-cuda-x86_64.zip.001
enfugue-server-0.2.3-win-cuda-x86_64.zip.002		 12.1.1 2.2.0.dev20230910
Windows CUDA+TensorRT enfugue-server-0.2.3-win-tensorrt-x86_64.zip.001
enfugue-server-0.2.3-win-tensorrt-x86_64.zip.002		 11.7.1 1.13.1
Linux CUDA enfugue-server-0.2.3-manylinux-cuda-x86_64.tar.gz.0
enfugue-server-0.2.3-manylinux-cuda-x86_64.tar.gz.1
enfugue-server-0.2.3-manylinux-cuda-x86_64.tar.gz.2		 12.1.1 2.2.0.dev20230910
Linux CUDA+TensorRT enfugue-server-0.2.3-manylinux-tensorrt-x86_64.tar.gz.0
enfugue-server-0.2.3-manylinux-tensorrt-x86_64.tar.gz.1
enfugue-server-0.2.3-manylinux-tensorrt-x86_64.tar.gz.2		 11.7.1 1.13.1

Linux

First, decide which version you want - with or without TensorRT support. TensorRT requires a powerful, modern Nvidia GPU.
Then, download the appropriate manylinux files here, concatenate them and extract them. A console command to do that is:

cat enfugue-server-0.2.3* | tar -xvz

You are now ready to run the server with:

./enfugue-server/enfugue.sh

Press Ctrl+C to exit.

Windows

Download the win64 files here, and extract them using a program which allows extracting from multiple archives such as 7-Zip.

If you are using 7-Zip, you should not extract both files independently. If they are in the same directory when you unzip the first, 7-Zip will automatically unzip the second. The second file cannot be extracted on its own.

Locate the file enfugue-server.exe, and double-click it to run it. To exit, locate the icon in the bottom-right hand corner of your screen (the system tray) and right-click it, then select Quit.

MacOS

Download the the macos file here, then double-click it to extract the package. When you run the application using the command below, your Mac will warn you of running downloaded packages, and you will have to perform an administrator override to allow it to run - you will be prompted to do this. To avoid this, you can run an included command like so:

./enfugue-server/unquarantine.sh

This command finds all the files in the installation and removes the com.apple.quarantine xattr from the file. This does not require administrator privilege. After doing this (or if you will grant the override,) run the server with:

./enfugue-server/enfugue.sh

Note: while the MacOS packages are compiled on x86 machines, they are tested and designed for the new M1/M2 ARM machines thanks to Rosetta, Apple's machine code translation system.

Upgrading

To upgrade any distribution, download and extract the appropriate upgrade package on this release. Copy all files in the upgrade package into your Enfugue installation directory, overwriting any existing files.

Installing and Running: Conda

To install with the provided Conda environments, you need to install a version of Conda.

After installing Conda and configuring it so it is available to your shell or command-line, download one of the environment files depending on your platform and graphics API.

  1. First, choose windows-, linux- or macos- based on your platform.
  2. Then, choose your graphics API:
    • If you are on MacOS, you only have access to MPS.
    • If you have a powerful next-generation Nvidia GPU (3000 series and better with at least 12 GB of VRAM), use tensorrt for all of the capabilities of cuda with the added ability to compile TensorRT engines. If you do not plan on using TensorRT, select cuda for the most optimized build for this API.
    • If you have any other Nvidia GPU or other CUDA-compatible device, select cuda.
    • Additional graphics APIs (rocm and directml) are being added and will be available soon.

Finally, using the file you downloaded, create your Conda environment:

conda env create -f <downloaded_file.yml>

You've now installed Enfugue and all dependencies. To run it, activate the environment and then run the installed binary.

conda activate enfugue
enfugue run

Installing and Running: Self-Managed Environment

If you would like to manage dependencies yourself, or want to install Enfugue into an environment to share with another Stable Diffusion UI, you can install enfugue via pip. This is the only method available for AMD GPU's at present.

pip install enfugue

If you are on Linux and want TensorRT support, execute:

pip install enfugue[tensorrt]

If you are on Windows and want TensorRT support, follow the steps detailed here.

Thank you!

Assets 19
Loading