HIP-1056 Add block streams design #9716
Conversation
Signed-off-by: Edwin Greene <[email protected]>
| ### Block protobuf to mirror node database mapping | ||
|
|
||
| #### Contract Create Transaction | ||
|
|
There was a problem hiding this comment.
More details on the missing entries still to come.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9716 +/- ##
============================================
- Coverage 92.32% 92.27% -0.06%
- Complexity 7413 7770 +357
============================================
Files 911 951 +40
Lines 31050 32457 +1407
Branches 3978 4114 +136
============================================
+ Hits 28667 29949 +1282
- Misses 1450 1544 +94
- Partials 933 964 +31 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
Signed-off-by: Edwin Greene <[email protected]>
|
| ``` | ||
|
|
||
| ### Database | ||
|
|
There was a problem hiding this comment.
Currently it is expected that there is a single round per block. The round has a round_number value that currently equals the block number. In the future multiple rounds could be included in a single block. I am not sure if we want to capture this somewhere.
There was a problem hiding this comment.
Rounds are sequential right? If so, can future proof by adding round_start, round_end columns.
|
|
||
| ## REST API | ||
|
|
||
| -Deprecate the REST API's State Proof Alpha. The record files and signature files will no longer be provided in the cloud buckets. |
There was a problem hiding this comment.
| -Deprecate the REST API's State Proof Alpha. The record files and signature files will no longer be provided in the cloud buckets. | |
| - Deprecate the REST API's State Proof Alpha. The record files and signature files will no longer be provided in the cloud buckets. |
|
|
||
| ### Data Flow | ||
|
|
||
|  |
There was a problem hiding this comment.
There's a StreamFileNotifier in between.
| ```java | ||
| package com.hedera.mirror.importer.downloader.block; | ||
|
|
||
| public class BlockStreamVerifier implements StreamFileNotifier, Closable { |
There was a problem hiding this comment.
The StreamFileNotifier is an existing component to batch and send stream files. The BlockStreamVerifier should leverage it, not implement it.
| public abstract class StreamPoller<StreamFile> { | ||
| public abstract void poll(); |
There was a problem hiding this comment.
Should prefer interfaces
| public abstract class StreamPoller<StreamFile> { | |
| public abstract void poll(); | |
| public interface StreamPoller<StreamFile> { | |
| void poll(); |
| ```java | ||
| package com.hedera.mirror.common.domain.transaction; | ||
|
|
||
| public class BlockItem implements StreamItem { |
There was a problem hiding this comment.
I assume this won't do any parsing like RecordItem currently does and instead just contain the Transaction, results, output, and state changes? Maybe we can show those fields and make it a simple record?
| ``` | ||
|
|
||
| ### Database | ||
|
|
There was a problem hiding this comment.
Rounds are sequential right? If so, can future proof by adding round_start, round_end columns.
| ```java | ||
| package com.hedera.mirror.importer.downloader.block; | ||
|
|
||
| public class BlockStreamVerifier implements StreamFileNotifier, Closable { |
There was a problem hiding this comment.
Closable is misspelled. But why is it needed? This should be a singleton bean.
| | name | Name of the blk file | | ||
| | node_id | | | ||
| | prev_hash | block_header.previous_block_hash | | ||
| | sidecar_count | Count of all transaction*output.\_transaction type*.sidecars in the blk file | |
There was a problem hiding this comment.
Should be zero. Sidecars should go away.
| | prev_hash | block_header.previous_block_hash | | ||
| | sidecar_count | Count of all transaction*output.\_transaction type*.sidecars in the blk file | | ||
| | size | byte size of the blk file | | ||
| | version | Record stream version - Perhaps a new version should be added to indicate "Translated from Block" | |
| | Database | Block Item | | ||
| | -------------- | ------------------------------------------------------ | | ||
| | entity.id | state_changes[i].state_change.map_update.value.topicId | | ||
| | entity.deleted | state_changes[i].state_change.map_update.value.deleted | |
There was a problem hiding this comment.
This value doesn't exist. This should be true i map_delete present or false for any other update.
Description:
Adds the design for phase one, support for block stream files.
Related issue(s):
Fixes #9573
Notes for reviewer:
Checklist