Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for event streaming API #417

Merged
merged 16 commits into from
Dec 20, 2023
Merged

Conversation

nvdtf
Copy link
Member

@nvdtf nvdtf commented Jun 28, 2023

Closes: #378

Note that this feature is still in development in access nodes. I'll update the examples before merging when it becomes available on all nodes.

@nvdtf nvdtf requested a review from peterargue June 28, 2023 21:13
@codecov-commenter
Copy link

codecov-commenter commented Jun 28, 2023

Codecov Report

Attention: 86 lines in your changes are missing coverage. Please review.

Comparison is base (20d217c) 61.25% compared to head (c4078aa) 62.57%.

Files Patch % Lines
access/grpc/grpc.go 73.97% 33 Missing and 5 partials ⚠️
access/grpc/convert.go 79.77% 24 Missing and 12 partials ⚠️
access/grpc/client.go 0.00% 6 Missing ⚠️
access/http/client.go 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #417      +/-   ##
==========================================
+ Coverage   61.25%   62.57%   +1.32%     
==========================================
  Files          26       26              
  Lines        3324     3658     +334     
==========================================
+ Hits         2036     2289     +253     
- Misses       1115     1180      +65     
- Partials      173      189      +16     
Flag Coverage Δ
unittests 62.57% <74.40%> (+1.32%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

access/client.go Outdated
GetExecutionDataByBlockID(ctx context.Context, blockID flow.Identifier) (*flow.ExecutionData, error)

// SubscribeExecutionData subscribes to execution data updates starting at the given block ID or height.
SubscribeExecutionData(ctx context.Context, startBlockID flow.Identifier, startHeight uint64) (<-chan flow.ExecutionDataStreamResponse, <-chan error, error)
Copy link
Member Author

@nvdtf nvdtf Jun 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be a good idea to break these into two functions: ByHeight and ById

Copy link
Contributor

@peterargue peterargue Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the rpc accepts either height/blockID which is a bit of an outlier for the Access API. All of the existing methods match the rpc. @sideninja @nvdtf what's your opinion on this?

  1. Keep it consistent with the rpc (keep it as-is in this PR)
  2. Keep it consistent with the other methods (add *ByBlockID, *ByHeight versions
  3. Align the protobuf with 2. If we go this route, I vote we do 1, then make those changes separately.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 2 is good enough. To me, accepting two fields and relying on either one is not good developer experience. Go SDK as the SDK can break this into two functions and call the RPC properly.
I like 3 as well, but don't want to push extra work for minor improvements.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated in 4275139

execution_data.go Show resolved Hide resolved
execution_data.go Show resolved Hide resolved
close: func() error { return conn.Close() },
jsonOptions: []json.Option{json.WithAllowUnstructuredStaticTypes(true)},
rpcClient: grpcClient,
executionDataClient: execDataClient,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the execution data API doesn't necessarily need to be run on the same interface as the access api. It's probably a good idea to allow specifying a different address/connection, maybe as a setting that overrides this original value.

access/grpc/convert.go Show resolved Hide resolved
access/grpc/grpc.go Outdated Show resolved Hide resolved
access/grpc/grpc.go Outdated Show resolved Hide resolved
access/grpc/grpc.go Outdated Show resolved Hide resolved
access/grpc/grpc.go Outdated Show resolved Hide resolved
access/grpc/grpc.go Outdated Show resolved Hide resolved
@peterargue
Copy link
Contributor

FYI: we moved the ExecutionDataAPI into its own namespace onflow/flow#1353

Looks like you're already mapping the import so there's probably no changes needed to this PR besides pulling in the latest version

@peterargue peterargue marked this pull request as ready for review November 28, 2023 04:41
@peterargue
Copy link
Contributor

@sideninja, @nvdtf mind taking a look? once this is ready, I can take a stab at adding it into the flow-cli

@nvdtf nvdtf requested a review from jribbink as a code owner December 14, 2023 14:07
@peterargue peterargue merged commit 58fb441 into master Dec 20, 2023
1 check passed
@peterargue peterargue deleted the navid/event-streaming branch December 20, 2023 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for new event streaming API
5 participants