Idea: Support running lightbus worker with WSGI/ASGI process

[adamcharnock]

This discussion on Hacker News got me thinking about the current necessity for the Lightbus worker to be run in its own process. Now Lightbus' threading has been sorted out, could we have the option of just running a Lightbus worker within the web process?

Why would anyone want to do this?

1. Pro: Simple sites become a single process (simpler deployment, less memory use)
2. Pro: Easier development workflow
3. Pro: Can use Lightbus where the number of processes is limited (i.e. Using Heroku's free dyno)

Why would this be a bad idea?

1. Con: High volume worker loads would impact serving of web requests
2. Con: Could increase Lightbus complexity
3. Con: More things to learn

Option 1 doesn't concern me, this is a fair trade-off which can be clearly explained. Option 2 concerns me more, and I would hope to mitigate this be (hopefully) avoiding having to modify the Lightbus core code. Option 3 is more concerning, I want to keep Lightbus simple and obvious, and this would be a wrinkle in the early tutorials.

How could this look?

Example WSGI for small site (using white noise for serving assets):

import os
from pathlib import Path

from django.core.wsgi import get_wsgi_application
from whitenoise import WhiteNoise

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_site.settings")

static_dir = Path(__file__).parent.parent / "staticfiles"

application = get_wsgi_application()
application = WhiteNoise(application, root=static_dir, prefix="static")

Adding in a lightbus worker could look something like this:

import os
from pathlib import Path

from django.core.wsgi import get_wsgi_application
from whitenoise import WhiteNoise
import lightbus.wsgi

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_site.settings")
os.environ.setdefault("LIGHTBUS_MODULE", "my_site.bus")
os.environ.setdefault("LIGHTBUS_SERVICE_NAME", "my_site")

static_dir = Path(__file__).parent.parent / "staticfiles"

application = get_wsgi_application()
application = WhiteNoise(application, root=static_dir, prefix="static")
application = lightbus.wsgi.Worker(application)

Thoughts

• If this is going to ease development as well as deployment, then it needs to plug into development servers and not just ASGI/WSGI, which could be a whole other framework-specific problem.

Want this?

If you want this leave a comment or give the issue a 👍 reaction