StorageDumper: make dump format compatible with StatesDumper

[AnnaShaleva]

Description

Problem:

#3281 removes StatesDumper plugin, however, the output format of state dump files generated by StorageDumper is incompatible with StatesDumper format. We have a set of existing tools that rely on output format of StatesDumper.

Note that it was said in #3271 (comment) that:

we have some tooling that uses StorageDumper output.

But it's not true, because all our tools use the output of StatesDumper actually.

Solution:

Keep the StatesDumper output format compatible with StorageDumper.

Type of change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

• Node synchronisation and resulting dump formats comparison.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[vncoelho]

We can adapt our tools, but check with NGD as well.

@NGDAdmin @superboyiii

[Wi1l-B0t]

Why change IReadOnlyStore to NeoSystem?
There is only one system.GetSnapshotCache()

[shargon]

The event receive NeoSystem

[superboyiii]

I strongly dislike this PR. I rarely say sth like this, but This PR is a real BUG. StorageDumper is StorageDumper, it's not StatesDumper. We remove StatesDumper because it has bug and we have a better choice->StorageDumper. Now we are going back... StorageDumper has clear format, each block data in a row, easy to read and compare. And it will not lose data when CLI is interrupted. But now see what we're doing... The bug is back again because some tool is not compatible... I don't understand why A PLUGIN has to be compatible with tools? Shouldn't tools adapt for plugin?
@AnnaShaleva You can try to install this Plugin and resync data. Randomly interrupt CLI for some times and export these dump data. You will see lots of data are lost.

[superboyiii]

[roman-khimov]

I don't understand why A PLUGIN has to be compatible with tools? Shouldn't tools adapt for plugin?

It's an API/format that is publicly available, so it's not that easy. Take RPC for example, it's a plugin, but this doesn't mean you can change its behavior easily, a lot of other code depends on it. This one of course is not that popular, but still.

Now if there is a real bug with the format (but I'd check if it's a format problem rather than implementation) we can consider switching to StatesDumper one, but IIRC it's not perfect either. This one is not release critical, we can solve it after 3.8 in any direction.

[vncoelho]

@AnnaShaleva ,create another plugin for now. We need to release soon, leave this plugin as it is. Later we decide how to merge.

[superboyiii]

I don't understand why A PLUGIN has to be compatible with tools? Shouldn't tools adapt for plugin?

It's an API/format that is publicly available, so it's not that easy. Take RPC for example, it's a plugin, but this doesn't mean you can change its behavior easily, a lot of other code depends on it. This one of course is not that popular, but still.

Now if there is a real bug with the format (but I'd check if it's a format problem rather than implementation) we can consider switching to StatesDumper one, but IIRC it's not perfect either. This one is not release critical, we can solve it after 3.8 in any direction.

At least we shouldn't bring bug back. Or I think you need is just making StatesDumper back. Then just revert it but don't change StorageDumper.