[draft] Split full update flow by components / linear components workflow #179
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Disclaimer: This is an incomplete draft of change, so some tests are not green yet (since not everything even implemented) and some code is messy, though main intention can be seen. Also I've intentionally haven't removed unused code for ease of review (it will be done in separate review for convenience).
Motivation
Make introducing changes in flow/components easier by simplifying flow, replacing bidirectional dependency with one-directional and more straightforward component update flow.
Make components more easy to test independently.
Improve ops experience of operator by introducing linear components flow.
Short version
Before: orchestrator sets statuses and components act according to them
After: components don't check any statuses but their own
Before: full update flow with mix of actions on ytsaurus and components sync
After: only order of components is set in the main loop
Before: order of component updates are dynamic
After: order of components update is declared in code
For now it is no sense to read all the change, since it is noisy.
It makes sense to check
Long version
I've removed so called "full update flow" (parts of it can be seen in sync.go and ytsaurus_client.go and components ). As a result main flow of components updating orchestration can be significantly simplified.
Of course things, that happened in the "full update flow" can't be thrown away, but they can be nicely placed in corresponding components.
So instead of:
we will have:
Why it is better? That way components doesn't need to check main resource status/update status and they become more loosely coupled with main loop and each other. Instead two way link (ytsaurus checking status of components <-> components checking state of ytsaurus) we have simple contract: component should report their status and main loop will scheduler sync when needed. So components became easier to write, because if their sync was called they must advance in their flow without checking anything other their own state. Also it is easier to test components separately example (quick and dirty) can be seen here.
Another consequence of change if we can easily implement linear order of updating components as described in #115. It can look somewhat like this.
Also with moving pieces of full update inside the componets I've redesigned their flow in a more declarative way, because 1) current version relied on update statuses, which are set from outside and code should be updated 2) current flow is not straightforward and hard to update it without breaking something.
To be done