Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change PluginMedian Interface #166

Closed
wants to merge 21 commits into from
Closed

Conversation

augustbleeds
Copy link
Contributor

@augustbleeds augustbleeds commented Sep 5, 2023

EDIT: This PR has been superceded by #509

Note: CI tests which build external repos will fail until "replace" is removed because replace is only respected in the top-level go.mod

Interface needs to be modified to include the gasPrice data source which tells us how much the gas price is on the chain during a report transmission (for chains which do not support onchain methods of retrieving gas price). It's part of the transmission structure similar to juelsPerFee so it's actually a value which is transmitted to the block-chain as a part of the median plugin

Example of it on Starknet

For now, I've made LOOPP median plugins have a nil value for the gasPrice data source, so it doesn't actually attempt to start a GRPC server. I think this should be ok for now because ocr will not call the method unless it is non-nil. Eventually, either in this PR or another, we will want to start a grpc service for the gas price data source EDIT: PR has been updated to serve gas price data source server if it exists

Related PRs:

  • PR Merge Order (from top to bottom in order)
    1. libocr
      • NumericalMedianFactory gets changed to have a gas price data source
      • Note: (If someone bumps libocr in core, median factory will need to be updated with nil)
      • Note: (You will not be able to bump libocr in chainlink-relay until the below PR gets merged)
    2. chainlink-common
      • Currently uses the replace directive, once libocr is merged we remove replace directive and bump the version of libocr
      • The plugin median interface is updated to include gas price data source
      • Begins the grpc service for gas price data source
      • Note: (You cannot bump chainlink-relay in chainlink repo until the PR below bets merged)
    3. chainlink-feeds
    4. chainlink
      • Uses replace directive for both chainlink-relay and libocr. They should get updated when the respective repos get updated
      • Parses TOML config for (optional) gas price data source and passes it into median factory.

@@ -17,7 +17,7 @@ type MedianProvider interface {

type PluginMedian interface {
// NewMedianFactory returns a new ReportingPluginFactory. If provider implements GRPCClientConn, it can be forwarded efficiently via proxy.
NewMedianFactory(ctx context.Context, provider MedianProvider, dataSource, juelsPerFeeCoin median.DataSource, errorLog ErrorLog) (ReportingPluginFactory, error)
NewMedianFactory(ctx context.Context, provider MedianProvider, dataSource, juelsPerFeeCoin, gasPrice median.DataSource, errorLog ErrorLog) (ReportingPluginFactory, error)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is gas price used for? That seems like a chain-specific detail that shouldn't leak out here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a data source which tells us how much the gas price is on the chain during a report transmission (for chains which do not support onchain methods of retrieving gas price). It's part of the transmission structure similar to juelsPerFee so it's actually a value which is transmitted to the block-chain as a part of the median plugin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all chains require it (like EVM chains), so if it is nil then ocr will not include it in a transmission

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why are these low-level chain-specific details leaking out of the relayer at all? Core and chain-agnostic product code shouldn't need to concern itself with these kinds of details.

Why is a pipeline being used to retrieve this data?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do other chain families all use the word gas though? I can't actually find any other terminology, so maybe nbd 🤷

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cosmos and ethereum L2s do. I believe solana operates differently and calls them "tx fees"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be unnecessary to include a data source that isn't used. That means we'd require node ops to include a data source that did nothing for all evm ocr2 pipelines.

The issue is precisely that whether or not to include a data source of gasPrice depends on the chain.

Can you explain how the gas price data source is going to be instantiated? We are actively working to put all chain-specific details in the relayer, and I want to make sure that we aren't taking a step back here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data source is another pipeline so it would be instantiated here https://github.com/smartcontractkit/chainlink/pull/10519/files#diff-7358f3249bf57875121f3bcb6211722cf2bee2f9b411dfff89a1974c48c5b67eR109.

A chain either has one or it doesn't so it's either nil or it's not. There's https://github.com/smartcontractkit/chainlink-relay/pull/166/files#diff-28ea9d6d17fb7499fbbd0f83828a8335ac28eacff629256d75b261bdb2ee123aR59 also when the loopp plugin serves the underlying data source. Would this be a step backward?

This suggests that whether or not to include it is chain-specific, so any behaviour related to it should happen in the relayer.

^ If data sources were to be instantiated in the relayer, that means configuration would also occur in the relayer. The infra doesn't exist, but if it were to exist you'd pass in pipeline configs from core to the relay and relay would instantiate them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is a pipeline being used to retrieve this data?

We reimburse transmit calls on chain. In order to do that correctly a reimbursement has to be calculated. This is easy in EVM where we can access tx.gasPrice in Solidity as well as measure gasleft() at the start and end of execution. But in basically every non-EVM chain this data isn't sourceable and we need to do two things:

  • come up with reasonable constants to approximate gas use based on lots and lots of transactions
  • include a medianized gasPrice in the report (we match the juelsPerFeeCoin source usage)

We were able to avoid this second one because:

  • on Terra the gas prices were fixed and rarely changed based on some external API price list
  • on Solana the TX cost was fixed (for now, but this changes with fee markets)

This gasPrice source is needed for both Starknet and Cosmos.

@augustbleeds augustbleeds temporarily deployed to integration September 5, 2023 15:31 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 5, 2023 15:31 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 5, 2023 15:31 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 6, 2023 21:47 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 6, 2023 21:47 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 6, 2023 21:47 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 8, 2023 15:00 — with GitHub Actions Inactive
@augustbleeds augustbleeds temporarily deployed to integration September 8, 2023 15:59 — with GitHub Actions Inactive
go.mod Outdated
@@ -65,6 +65,8 @@ require (
gopkg.in/yaml.v3 v3.0.1 // indirect
)

replace github.com/smartcontractkit/libocr => github.com/augustbleeds/libocr v0.0.0-20230901193648-78ff9454fecd
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

temp change: will get removed and bumped when libocr gets updated

@augustbleeds augustbleeds changed the title DRAFT: Change PluginMedian Interface Change PluginMedian Interface Sep 8, 2023
@augustbleeds augustbleeds temporarily deployed to integration September 8, 2023 21:01 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants