Issue 49822: Pipeline does not show a job as failed if DB fails #5363
Conversation
…to secondary server
| { | ||
| Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler(); | ||
|
|
||
| // Get configured quartz Trigger from subclass |
There was a problem hiding this comment.
I'm not understanding how the comment aligns with the code. This doesn't seem to be getting anything from the subclass.
There was a problem hiding this comment.
I failed to clean up the comments from the message digest code that I started with. Fixed.
| // Attempt the updates to the DB rows again in the hopes that the DB is back online | ||
| synchronized (_queuedUpdates) | ||
| { | ||
| if (!_queuedUpdates.isEmpty()) |
There was a problem hiding this comment.
The if statement doesn't seem necessary.
There was a problem hiding this comment.
I restructured the code a bit in my latest comment and the if check is a little more useful now. Not vital, but it saves a tiny bit of work
| // Then try based on file path | ||
| sfExist = getStatusFile(job.getContainer(), job.getLogFilePath()); | ||
| } | ||
| PipelineStatusFileImpl sfExist = getJobStatusFile(job.getJobGUID()); |
There was a problem hiding this comment.
I know this was existing code and likely not worth changing now, but the name sfExist here is odd to me. existingFile would read better to me.
There was a problem hiding this comment.
I agree but am opting to keep the diff smaller instead of doing the rename
| // This will happen if job was previously run from a cloud root. | ||
| sfExist = getStatusFile(job.getContainer(), job.getRemoteLogPath()); | ||
| } | ||
| PipelineStatusFileImpl sfSet = new PipelineStatusFileImpl(job, status, info); |
There was a problem hiding this comment.
sfSet is also a strange name to me, to the extent that I don't know what a better name is.
| else if (PipelineJob.TaskStatus.complete.matches(status)) | ||
| { | ||
| // Make sure the Enterprise Pipeline recognizes this as a completed | ||
| // job, even if did it not have a TaskPipeline. |
There was a problem hiding this comment.
| // job, even if did it not have a TaskPipeline. | |
| // job, even if it did not have a TaskPipeline. | |
Rationale
If a pipeline job is running or queued to be run when the database goes down, we fail to update its state in the DB. If the DB later comes back online, the job looks like it's still running or queued. That's inaccurate and can lead to ETLs not being queued on schedule, etc.
Changes