Terrible gradio performance

[ClaraSanders]

I have many problems running colab on gradio servers under local and normal. It is especially infuriating how the Generate button stops working after just two or three generations. This happened at the beginning to the point that I was about to give up on stable diffusion, but somehow this issue disappeared over time, but now is back with a vengeance. Am I the only one who experiences this issue? It is my browser? It is just that gradio´s servers are trash? Is there any alternative to gradio?

[THX1139b]

I haven't been able to get Automatic1111 to work since about a month ago. Typically I manage to generate a couple of tests and then nothing more is happening when hitting the generate button. The status bar is the first to go and quickly thereafter everything stops responding. I've given up on using it for further work on an image with img2img or ControlNet. It works for a while, on a good day, for generating text to images.

I was adviced to try RunPod, which I have now finally learnt how to set up and use. But the problem is there as well, unrelated to how much GPU power I rent. Usually the interface stops responding after a few minutes. SD that isn't run locally installed seems to hit a wall somewhere and then just go into an error loop. (I'm wondering if it has something to do with ControlNet being installed.)

Sometimes I can restart the UI and keep going but usually only for a short while.

[TheLastBen]

did you try ngrok ?

[ClaraSanders]

I just signed up and tried it. It performs a little better than gradio. Now it only freezes if you load a model various times.
However when I start the last cell using ngrok I get this warning (although it loads the model and runs the server)

/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/init.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6StatusC1EN10tensorflow5error4CodeESt17basic_string_viewIcSt11char_traitsIcEENS_14SourceLocationE']
warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/init.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']
warnings.warn(f"file system plugins are not loaded: {e}")

[SDPDVADER]

Using ngrok and got this /usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/init.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6StatusC1EN10tensorflow5error4CodeESt17basic_string_viewIcSt11char_traitsIcEENS_14SourceLocationE']
warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/init.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ['/usr/local/lib/python3.9/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']
warnings.warn(f"file system plugins are not loaded: {e}")

[TheLastBen]

that's a message related to colab, the notebook isn't the cause

[HaikenEdge]

I've been experiencing this as well; I generally generate one image as a sample before generating them in bulk, and I've found that I've had to basically re-run the Start Stable-Diffusion section of the notebook after every bulk process.

I'm glad it still works, so this isn't a criticism of the notebook. I'm just trying to offer a solution even if it takes a little bit longer than it used to.

[TheLastBen]

use the latest notebook

[ClaraSanders]

I reinstalled the notebook from scratch and the only difference is that the models now load much faster but the notebook still freezes after two or three generations. Don't worry. It is what it is. At least there aren't any installation errors for now.

[HaikenEdge]

I was having a similar problem until I turned off the Cloudflare tunnel. Now, the Generate button no longer stops working for me.

But that's just anecdotal evidence.

[ClaraSanders]

Hey, you are right. It works fine now after turning off Cloudflare tunnel. So if you are reading this Ben, please leave it the way it is now.

[Headupdef]

Error completing request
Arguments: ('task(5oj4xpefq3ar24c)', 'looking at viewer, bare shoulders, a closeup portrait of a playful teacher, mature woman, undercut hair, (floral blouse), (pencil skirt), amazing body, pronounced feminine feature, busty, classroom, [| ginger | auburn hair], freckles, flirting with camera, FreckledVixon, lora:FreckledVixon:0.8, centered, realistic, professional,', 'bad-picture-chill-75v bad-artist bad_prompt_version2 bad-hands-5, child, kid (low resolution:1.4), ((blurry:1.3)), (worst quality:1.3), (low quality:1.3), easynegative', [], 35, 0, True, False, 1, 1, 7, None, -1.0, 0, 0, 0, False, 1024, 768, False, 0.7, 2, 'Latent', 0, 0, 0, [], 0, <scripts.external_code.ControlNetUnit object at 0x7fcf7e03f190>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', 0, '', 0, '', True, False, False, False, 0, None, False, 50) {}
Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 76, in alternate
yield next(args[(step - 1)%len(args)])
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/lark/visitors.py", line 116, in _call_userfunc
return f(children)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 84, in start
return ''.join(flatten(args))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 83, in flatten
yield from flatten(gen)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 83, in flatten
yield from flatten(gen)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 82, in flatten
for gen in x:
RuntimeError: generator raised StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/txt2img.py", line 56, in txt2img
processed = process_images(p)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 486, in process_images
res = process_images_inner(p)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 626, in process_images_inner
c = get_conds_with_caching(prompt_parser.get_multicond_learned_conditioning, prompts, p.steps, cached_c)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 570, in get_conds_with_caching
cache[1] = function(shared.sd_model, required_prompts, steps)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 205, in get_multicond_learned_conditioning
learned_conditioning = get_learned_conditioning(model, prompt_flat_list, steps)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 129, in get_learned_conditioning
prompt_schedules = get_learned_conditioning_prompt_schedules(prompts, steps)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 102, in get_learned_conditioning_prompt_schedules
promptdict = {prompt: get_schedule(prompt) for prompt in set(prompts)}
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 102, in
promptdict = {prompt: get_schedule(prompt) for prompt in set(prompts)}
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 100, in get_schedule
return [[t, at_step(t, tree)] for t in collect_steps(steps, tree)]
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 100, in
return [[t, at_step(t, tree)] for t in collect_steps(steps, tree)]
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/prompt_parser.py", line 90, in at_step
return AtStep().transform(tree)
File "/usr/local/lib/python3.9/dist-packages/lark/visitors.py", line 153, in transform
return self._transform_tree(tree)
File "/usr/local/lib/python3.9/dist-packages/lark/visitors.py", line 149, in _transform_tree
return self._call_userfunc(tree, children)
File "/usr/local/lib/python3.9/dist-packages/lark/visitors.py", line 120, in _call_userfunc
raise VisitError(tree.data, tree, e)
lark.exceptions.VisitError: Error trying to process rule "start":

generator raised StopIteration