Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Supervisor's autostart setting to control which processes start in which environment #249

Open
seanh opened this issue Nov 2, 2022 · 0 comments

Comments

@seanh
Copy link
Contributor

seanh commented Nov 2, 2022

Not all of h-periodic's beat processes are needed in every environment that h-periodic is deployed to. For example the ca-central-1 deployment of h-periodic doesn't need the checkmate_beat process.

h-periodic currently uses a Python hack to avoid starting the checkmate_beat process if the DISABLE_CHECKMATE_BEAT envvar is set:

if asbool(environ.get("DISABLE_CHECKMATE_BEAT")): # pragma: nocover
print("checkmate_beat disabled by DISABLE_CHECKMATE_BEAT environment variable")
sys.exit()

The h_beat and lms_beat have the same hack.

I think Supervisor may even restart these processes when they exit! Though I think by default it tries a fixed number of times then gives up, it doesn't keep restarting them forever. I think the restarting can be completely disabled in supervisord.conf but that's not ideal: when a process is actually supposed to be running then we actually do want Supervisor to restart it if it crashes.

The right way to do this is by using envvars to control each process's autostart setting in supervisord.conf, as has been done in h:

[program:nginx]
command=nginx
stdout_logfile=NONE
stderr_logfile=NONE
stdout_events_enabled=true
stderr_events_enabled=true
autostart = %(ENV_ENABLE_NGINX)s


[program:web]
command=newrelic-admin run-program pserve conf/app.ini
stdout_logfile=NONE
stderr_logfile=NONE
stdout_events_enabled=true
stderr_events_enabled=true
autostart = %(ENV_ENABLE_WEB)s


[program:websocket]
command=pserve %(ENV_WEBSOCKET_CONFIG)s
stdout_logfile=NONE
stderr_logfile=NONE
stdout_events_enabled=true
stderr_events_enabled=true
autostart = %(ENV_ENABLE_WEBSOCKET)s


[program:worker]
command=newrelic-admin run-program hypothesis celery worker --loglevel=INFO
stdout_logfile=NONE
stderr_logfile=NONE
stdout_events_enabled=true
stderr_events_enabled=true
autostart = %(ENV_ENABLE_WORKER)s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant