workflow_index.update_time NOT NULL constraint breaking upgrade to 3.21.5

[lbestatlas]

Pull Request type

• Bugfix
• Feature
• Refactoring (no functional changes, no api changes)
• Build related changes
• WHOSUSING.md
• Other (please describe):

NOTE: Please remember to run ./gradlew spotlessApply to fix any format violations.

Changes in this PR

Address items raised in issue 237 conductor-oss#237, these issues were:

1. When running the postgres v13 flyway upgrade we have found some workflows without update_time in the json_data column, which causes update_time to be null when the data is back filled in the v13 script. The subsequent application of the NOT NULL constraint fails.
2. When running old and new versions of conductor concurrently during rollout, workflows executed in the old version have failures indexing workflows as workflow_index.update_time is NULL
3. The new code does not set the update_time in the case of an update only an insert, so this seems incorrect if the intention was to prevent out of sequence updates to the workflow_index.
4. The back-fill script on the workflow_index can potentially lock up the table for minutes in environments where there is a large number of workflows indexed.
5. Also, the back-fill script time format incorrectly removes precision from the milliseconds component.

To fix these issues this PR make the following changes:

1. Replaces the v13 flyway migration script with V13.1 that adds update_time with a default of 'epoch' if the column doesn't exist, and sets the default of 'epoch' if the column does exist. Adding a default in postgres is inexpensive, as it does not update existing rows. The default means the constraint can be added without running the back-fill and stops the issues when have old and new versions running at the same time.

2. Fix the PostgresIndexDao.indexWorkflow to also set the update_time during an UPDATE. This will ensure updateTime accurately reflects the most recent write.

3. Address the issues with the back-fill script logic, to ensure it doesn't try to insert NULL values and correctly formats the timestamp.

4. Make the back-fill script for update_time optional, so that expensive data migrations do not need to be run at start up.

Alternatives considered

I considered removing the not null constraint. However, this meant more complications dealing with null values in the consuming code. So given a default is inexpensive to add this seemed preferable.

I also considered dropping the back-fill script entirely, as it doesn't appear to impact any logic. However, I decided to add the ability to disable data migrations, for expensive scripts. Flyway community edition does not support https://flywaydb.org/blog/skipExecutingMigrations so it is impossible to defer scripts from being run during the migration if using that edition.