Log message of state includes the FROM-state but TO-state is sometimes empty

[tobiscr]

Description

KIM is logging state changes. Two things should be improved:

1. The TO-state is sometimes empty, example:

{"level":"info","ts":"2024-11-29T12:12:21Z","msg":"switching state from github.com/kyma-project/infrastructure-manager/internal/controller/runtime/fsm.sFnWaitForShootReconcile.updateStatusAndRequeueAfter.sFnUpdateStatus.func2 to ","runtimeID":"i539990-fd2b-4186-b0e5-8a1bc8ede01","shootName":"i539990-test","requestID":3213,"result":{"Requeue":false,"RequeueAfter":30000000000},"err":null,"mFnIsNill":true}

{"level":"info","ts":"2024-11-29T12:10:57Z","msg":"switching state from github.com/kyma-project/infrastructure-manager/internal/controller/runtime/fsm.sFnEmmitEventfunc.func1 to ","runtimeID":"eb05f922-f2c1-4436-a506-745a7feacdcb","shootName":"d7ce229","requestID":3210,"result":null,"err":null,"mFnIsNill":true}

2. Loglevel "INFO" should be challenged as it indicates a relevant event which is relevant for maintainers/administrators. I agree that state-changes could be helpful for debugging, but the generated noise is too much (consider 1.500 Clusters will be reconcilled - this will generate several GB logs per day). Options I see:
   1. Do we have to track each state-change separately - isn't it possible to track them whole chain of changes in one message?
   2. If it's not possible, we shouild consider to log state-changes with DEBUG to avoid to many noise.

Expected result

1. TO-state is never empty
2. State-chagnes are optimised or reduced to DEBUG-level to reduce "log noise"

Actual result

1. Empty TO-changes are printed when the FROM-state is updateStatusAndRequeueAfter.sFnUpdateStatus.func2 or fsm.sFnEmmitEventfunc.func1
2. Log-Noise is very high caused by logging all details of state-changes

Steps to reproduce

Review logs of a reconciled cluster in KIM

Troubleshooting