Merge pull request #74 from noskill/docsbr

more docs for pipes

docs/api.md

@@ -6,6 +6,15 @@
 Pipes classes implement different ways to generate 
 or process images using diffusion models.
 
+
+All Pipe classes have two methods: *setup* and *gen*.
+
+*setup*'s purpuse is define parameters of the pipeline for image generation. Pipelines that take an image as an 
+input perform image preprocessing in *setup*. Setup's arguments are stored in the pipeline and used whenever *gen* method is used.
+
+*gen* takes as an input parameters are not stored inside the pipeline. All the pipelines take a dictionary as an input.
+Expected parameters are prompt(str), generator (torch.Generator), negative_prompt (str).
+
 **Prompt2ImPipe** is a pipe that generates an image from text prompt.
 
 
@@ -16,6 +25,20 @@ image = pipe.gen({'prompt': 'bio-tech lab with computers and exotic flowers, art
 image.save('bio.png')
 ```
 
+*setup* parameters
+
+width - width of image to generate  
+heigth - height of image to generate  
+guidance_scale - strength of the prompt influence on generation process.  
+steps - The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. Default value is 50.  
+clip_skip - Number of layers to be skipped from CLIP while computing the prompt embeddings. Skipping some layers gives less precise representation of the prompt. Default value is 0.  
+
+Optimal values of guidance_scale and steps vary a lot between different checkpoints.
+
+
+*gen* parameters
+
+
 ***Im2ImPipe** is a pipe that generates an image from another image.
 
 ```
@@ -25,20 +48,39 @@ img = pipe.gen({'prompt': 'biolab, anime style drawing, green colors'})
 img.save('bio1.png')
 ```
 
+*setup* parameters
+
+fimage - File path to the input image.  
+image - Input image. Can be used instead of fimage.  
+strength - Strength of image modification. Defaults to 0.75. A lower strength values keep result close to the input image. value of 1 means input image more or less ignored.  
+scale -  The scale factor for resizing of the input image. The output image will have dimentions (height * scale, width * scale) Defaults to None.  
+guidance_scale, steps, clip_skip - same as in Prompt2ImPipe  
+
+
 **Cond2ImPipe** is a pipe that generates 
-an image from another image plus conditioning 
-image e.g. image after canny edge detection etc.  
+an image from a special conditioning 
+image e.g.image after canny edge detection, or etc.  
 Conditioning image is processed internally with controlnet and  
 uses StableDiffusion(XL)ControlNetPipeline
 
 Models are expected to be in ./models-cn/ for StableDiffusion 
 and in ./models-cn-xl/ for StableDiffusionXL
 
-**CIm2ImPipe** is similiar to Cond2ImPipe.  
+
+**CIm2ImPipe** is a subclass of Cond2ImPipe.  
+
 The difference is that the conditional image is not
 taken as input but is obtained from the input image, which
-should be processed, and the image processor 
-depends on the conditioning type.
+is processed internally by the image processor. The image processor   
+depends on the conditioning type specified in the constructor.
+
+
+
+*setup* parameters
+
+fimage, image - same as in Im2ImPipe
+cscales - strength of control image influence  
+width, height, steps, clip_skip, guidance_scale - same as in Prompt2ImPipe
 
 ```
 model_id = 'runwayml/stable-diffusion-v1-5'
@@ -81,6 +123,15 @@ img = pipe.gen({'prompt': prompt, 'seed':84958344})
 img.save('inpaint1.png')
 ```
 
+*setup* parameters
+
+original_image - image without mask
+image_painted - modified version of original_image, this parameter should be skipped if mask is passed.  
+mask - The mask. Defaults to None. If None it will be computed from the difference 
+between original_image and image_painted, should be skipped if image_painted is passed  
+blur - The blur radius for the mask to apply for generation process.  
+blur_compose - The blur radius for composing the original and generated images.  
+scale - The scale factor for resizing of the input image. The output image will have dimentions (height * scale, width * scale)
 
 ## metafusion service
 

docs/generated/multigen.html

@@ -19,12 +19,24 @@
 <tr><td bgcolor="#aa55cc"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
 <td width="100%"><table width="100%" summary="list"><tr><td width="25%" valign=top><a href="multigen.hypernet.html">hypernet</a><br>
 <a href="multigen.loader.html">loader</a><br>
+<a href="multigen.log.html">log</a><br>
+<a href="multigen.lpw_stable_diffusion.html">lpw_stable_diffusion</a><br>
+</td><td width="25%" valign=top><a href="multigen.lpw_stable_diffusion_xl.html">lpw_stable_diffusion_xl</a><br>
+<a href="multigen.lpwxl.html">lpwxl</a><br>
 <a href="multigen.pipelines.html"><strong>pipelines</strong>&nbsp;(package)</a><br>
-</td><td width="25%" valign=top><a href="multigen.pipes.html">pipes</a><br>
+<a href="multigen.pipes.html">pipes</a><br>
+</td><td width="25%" valign=top><a href="multigen.prompt_helper.html">prompt_helper</a><br>
 <a href="multigen.prompting.html">prompting</a><br>
 <a href="multigen.sessions.html">sessions</a><br>
-</td><td width="25%" valign=top><a href="multigen.util.html">util</a><br>
-<a href="multigen.worker.html">worker</a><br>
+<a href="multigen.util.html">util</a><br>
+</td><td width="25%" valign=top><a href="multigen.worker.html">worker</a><br>
 <a href="multigen.worker_base.html">worker_base</a><br>
-</td><td width="25%" valign=top></td></tr></table></td></tr></table>
+</td></tr></table></td></tr></table><p>
+<table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
+<tr bgcolor="#55aa55">
+<td colspan=3 valign=bottom>&nbsp;<br>
+<font color="#ffffff" face="helvetica, arial"><big><strong>Data</strong></big></font></td></tr>
+
+<tr><td bgcolor="#55aa55"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
+<td width="100%"><strong>hints</strong> = {'author_style': ['Jean-Baptiste Monge style', 'styled in Art Nouveau', 'painting by Ed Blinkey', 'by Atey Ghailan', 'by Jeremy Mann', 'by Greg Manchess', 'by Antonio Moro', 'painting art by greg rutkowski', 'in the style of kandinsky', 'in the style of guillem h. pongiluppi', 'by Abigail Larson', 'by John Sloane', 'in the style of John Berkey', 'by Russ Mills', 'by George Callaghan', 'by Ian Miller', 'by Luke Fildes', 'inspired by Chris Achilleos', 'by James Jean', 'style of Wayne Barlowe', ...], 'colors': ['pastel colors', 'bioluminescens color', 'vibrant palette', 'colorful', 'dreamlike colors', 'neon lights', 'black and white', 'monochrome', 'unusual colors', 'complementary colors', 'flowing colors', 'dark', 'bright', '24bit colors'], 'effects': ['strong environmental light', ['cinema lighting', 'cinematic light'], 'haze lighting', 'reflections', 'light leaks', 'detailed shading', 'soft shadows', 'sheen', 'glowing', 'dynamic motion'], 'quality': [['ultra resolution', 'ultra high res', 'highest resolution', 'absurdres'], ['hyperrealistic', 'detailed hyperrealism'], 'perfectly balanced', 'score_9', '35mm photography', ['8k RAW photo', '8k uhd', 'cg unity 8k wallpaper', '16k hdri', '32k UHD'], ['masterpiece', 'famous artwork'], ['trending on ArtStation', 'trending on CGSociety'], ['super quality', 'best quality', 'extreme quality', 'top quality'], ['highly detailed', 'high detail', 'ultra-detailed']], 'shooting': ['double exposure', 'extremely sharp focus', 'large depth of field', 'deep depth of field', ['artistic blur', 'bokeh'], 'soft focus', 'cinematic shot', ['35mm photo', '50mm photo'], 'dynamic angle', 'film grain'], 'style': ['surrealism', ['symbolism', 'cubism'], 'psychedelic art', ['impressionism', 'expressionism', 'neo-expressionist'], 'neoclassicism', ['cyberpunk', 'urbanpunk', 'cyberpunk mood', 'cyberpunk rave'], ['snthwve style', 'nvinkpunk style', 'ink punk style'], ['art photography', 'artistic photorealism'], ['professional photo', 'iphone photo'], 'poster art', 'edgy street art', ['afrofuturism style', 'in the style of chinapunk'], ['sci-fi concept art', 'scifi style', 'futuristic'], 'painterly style', 'pixar style', 'quilling', ['pencil sketch', 'pencil painting'], 'watercolor painting'], 'style_modifiers': ['geometric patterns', ['fusion of art styles', 'concept art', 'fine art'], 'grafitti', 'splash', 'modern ink', ['multi-layered collages', 'abstract multilayer texture'], 'iconic album covers', 'fragmented icons', 'stylized photo', 'cinematic illustration', 'dripping paint', 'splatter art', 'fractal art', 'oil on canvas', ['digital painting', 'digital dreamscape'], 'holographic style'], 'vibe': ['energy-filled illustration', 'intricate', 'vibrant', 'stunning', 'aesthetic', 'sensual', 'radiant vibe']}</td></tr></table>
 </body></html>

docs/generated/multigen.loader.html

@@ -17,9 +17,15 @@
 <font color="#ffffff" face="helvetica, arial"><big><strong>Modules</strong></big></font></td></tr>
 
 <tr><td bgcolor="#aa55cc"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
-<td width="100%"><table width="100%" summary="list"><tr><td width="25%" valign=top><a href="logging.html">logging</a><br>
-</td><td width="25%" valign=top><a href="torch.html">torch</a><br>
-</td><td width="25%" valign=top></td><td width="25%" valign=top></td></tr></table></td></tr></table><p>
+<td width="100%"><table width="100%" summary="list"><tr><td width="25%" valign=top><a href="copy.html">copy</a><br>
+<a href="diffusers.html">diffusers</a><br>
+</td><td width="25%" valign=top><a href="logging.html">logging</a><br>
+<a href="psutil.html">psutil</a><br>
+</td><td width="25%" valign=top><a href="random.html">random</a><br>
+<a href="sys.html">sys</a><br>
+</td><td width="25%" valign=top><a href="threading.html">threading</a><br>
+<a href="torch.html">torch</a><br>
+</td></tr></table></td></tr></table><p>
 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
 <tr bgcolor="#ee77aa">
 <td colspan=3 valign=bottom>&nbsp;<br>
@@ -46,11 +52,17 @@
 <td width="100%">Methods defined here:<br>
 <dl><dt><a name="Loader-__init__"><strong>__init__</strong></a>(self)</dt><dd><tt>Initialize&nbsp;self.&nbsp;&nbsp;See&nbsp;help(type(self))&nbsp;for&nbsp;accurate&nbsp;signature.</tt></dd></dl>
 
-<dl><dt><a name="Loader-get_pipeline"><strong>get_pipeline</strong></a>(self, model_id)</dt></dl>
+<dl><dt><a name="Loader-cache_pipeline"><strong>cache_pipeline</strong></a>(self, pipe: diffusers.pipelines.pipeline_utils.DiffusionPipeline, model_id)</dt></dl>
+
+<dl><dt><a name="Loader-clear_cache"><strong>clear_cache</strong></a>(self, device)</dt></dl>
+
+<dl><dt><a name="Loader-from_pipe"><strong>from_pipe</strong></a>(self, cls, pipe, additional_args)</dt></dl>
+
+<dl><dt><a name="Loader-get_gpu"><strong>get_gpu</strong></a>(self, model_id) -&gt; List[int]</dt><dd><tt>return&nbsp;list&nbsp;of&nbsp;gpus&nbsp;with&nbsp;loaded&nbsp;model</tt></dd></dl>
 
-<dl><dt><a name="Loader-load_pipeline"><strong>load_pipeline</strong></a>(self, cls: Type[diffusers.pipelines.pipeline_utils.DiffusionPipeline], path, torch_dtype=torch.float16, device=device(type='cuda'), **additional_args)</dt></dl>
+<dl><dt><a name="Loader-get_pipeline"><strong>get_pipeline</strong></a>(self, model_id, device=None)</dt></dl>
 
-<dl><dt><a name="Loader-register_pipeline"><strong>register_pipeline</strong></a>(self, pipe: diffusers.pipelines.pipeline_utils.DiffusionPipeline, model_id)</dt></dl>
+<dl><dt><a name="Loader-load_pipeline"><strong>load_pipeline</strong></a>(self, cls: Type[diffusers.pipelines.pipeline_utils.DiffusionPipeline], path, torch_dtype=torch.float16, device=None, **additional_args)</dt></dl>
 
 <dl><dt><a name="Loader-remove_pipeline"><strong>remove_pipeline</strong></a>(self, model_id)</dt></dl>
 
@@ -69,14 +81,19 @@
 <font color="#ffffff" face="helvetica, arial"><big><strong>Functions</strong></big></font></td></tr>
 
 <tr><td bgcolor="#eeaa77"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
-<td width="100%"><dl><dt><a name="-copy_pipe"><strong>copy_pipe</strong></a>(pipe)</dt></dl>
+<td width="100%"><dl><dt><a name="-awailable_ram"><strong>awailable_ram</strong></a>()</dt></dl>
+ <dl><dt><a name="-copy_pipe"><strong>copy_pipe</strong></a>(pipe)</dt></dl>
+ <dl><dt><a name="-count_params"><strong>count_params</strong></a>(model)</dt></dl>
+ <dl><dt><a name="-get_model_size"><strong>get_model_size</strong></a>(pipeline)</dt></dl>
+ <dl><dt><a name="-get_size"><strong>get_size</strong></a>(obj)</dt></dl>
 </td></tr></table><p>
 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
 <tr bgcolor="#55aa55">
 <td colspan=3 valign=bottom>&nbsp;<br>
 <font color="#ffffff" face="helvetica, arial"><big><strong>Data</strong></big></font></td></tr>
 
 <tr><td bgcolor="#55aa55"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
-<td width="100%"><strong>Type</strong> = typing.Type<br>
+<td width="100%"><strong>List</strong> = typing.List<br>
+<strong>Type</strong> = typing.Type<br>
 <strong>logger</strong> = &lt;Logger /home/imgen/projects/metafusion/multigen/loader.py (WARNING)&gt;</td></tr></table>
 </body></html>