You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I use celery to process some raw data and load it into the database every day. On some occasions, the execution of the task takes far longer than typical. There are also instances where I kick the task off manually, outside of the usual schedule. In either instance, there are times where the current running instance overlaps with the scheduled start of the same task, and they end up stepping on each other in the database.
I could probably handle this outside of celery easy enough, just add a table to the database with a flag that indicates if I am currently processing or not, then update that when I start and end a job, and wait for it if it is already set. But before I go that route, is this possible through beat?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I use celery to process some raw data and load it into the database every day. On some occasions, the execution of the task takes far longer than typical. There are also instances where I kick the task off manually, outside of the usual schedule. In either instance, there are times where the current running instance overlaps with the scheduled start of the same task, and they end up stepping on each other in the database.
I could probably handle this outside of celery easy enough, just add a table to the database with a flag that indicates if I am currently processing or not, then update that when I start and end a job, and wait for it if it is already set. But before I go that route, is this possible through beat?
Beta Was this translation helpful? Give feedback.
All reactions