Shutdown vs leaseLost and shardEnded

[gavinvangent]

Hi Team,

Version 1 of this library required a shutdown function to be exposed, which v2 replaced with leaseLost and shardEnded. Having migrated to v2, I went through the process of implementing the new leaseLost and shardEnded functions and deployed the code (a few months ago). All went well until i had to split the shards which caused more than an hour of downtime while hunting for reasons why we weren't able to get beyond the shard split.

Turns out, v2 appears to be checking for the shutdown function and, if present, invokes it before doing any sort of checks for the new functions first. In my case, I rerouted the new leaseLost and shardEnded functions into the shutdown function I had before the v2 migration (with slight modifications):

leaseLost(_leaseLostInput: kcl.LeaseLossInput, completeCallback: kcl.Callback): void {
  return this.shutdown('LeaseLost', undefined, completeCallback);
}

shardEnded(shardEndedInput: kcl.ShardEndedInput, completeCallback: kcl.Callback): void {
  return this.shutdown('ShardEnded', shardEndedInput.checkpointer, completeCallback);
}

shutdown(reason: string, checkpointer: kcl.Checkpointer, completeCallback: kcl.Callback): void {
  ...
}

Having a shutdown function on my processor resulted in the kcl not invoking the shardEnded function and instead called my shutdown function, with my reason actually being a checkpointer, and my checkpointer was actually the completeCallback function. This failed horribly.

I suggest reworking this part of the codebase so that it attempts to call the new functions first, and if the new functions aren't present, then revert to attempting the shutdown function.