Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protocol fee adjustment in driver #2213

Merged
merged 38 commits into from
Jan 11, 2024
Merged

Protocol fee adjustment in driver #2213

merged 38 commits into from
Jan 11, 2024

Conversation

sunce86
Copy link
Contributor

@sunce86 sunce86 commented Dec 26, 2023

Description

Replacement PR for #2097

As @fleupold suggested, protocol fee adjustments are done fully in driver, in class Fulfillment. It goes like this:

  1. Solver responds with solution (accounts for surplus_fee only)
  2. Driver creates the fulfillment (executed amount + surplus_fee) based on (1)
  3. Driver adjusts fulfillment for protocol fees. It
  • calculates protocol fee
  • reduces executed amount by protocol fee.
    (note that this is basically the same as increasing surplus_fee)
  1. Driver encodes the fulfillment using the SettlementEncoder functionality, meaning custom price adjustments will be reused for protocol fee also.

How to test

Added unit tests for

  1. sell limit order fok
  2. buy limit order fok
  3. sell limit order partial (with 3 fulfillments)
    (but now removed to keep the domain clean)

@sunce86 sunce86 added the E:4.2 Protocol Fee Implementation See https://github.com/cowprotocol/pm/issues/29 for details label Dec 26, 2023
@sunce86 sunce86 requested a review from a team as a code owner December 26, 2023 22:54
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel comfortable adding this large chunk of core logic right into the existing fulfilment constructor.
Can we create a fee specific domain module (domain/competition/solution/fee.rs) and add an extension impl Fulfilment there which exposes an apply_fee(self, ...) -> Self that gets triggered from the solution (either upon constructing or before being encoded)?
This module can then encapsulate all fee related logic.

As for testing, I think one nice side effect of the examples you worked on with Olga and Anxo is that they can serve as ideal e2e tests, or at least driver integration tests, which is how we want to test core functionality going forward (instead of unit tests).

crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/order/mod.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/infra/solver/dto/solution.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
@sunce86
Copy link
Contributor Author

sunce86 commented Dec 27, 2023

I don't feel comfortable adding this large chunk of core logic right into the existing fulfilment constructor.
Can we create a fee specific domain module (domain/competition/solution/fee.rs) and add an extension impl Fulfilment there which exposes an apply_fee(self, ...) -> Self that gets triggered from the solution (either upon constructing or before being encoded)?
This module can then encapsulate all fee related logic.

Tried this already but disliked the fact that the Fulfillment object has two states and I would have to make field Fulfillment::protocol_fee optional. Since you prefer to not even have this field, then this is doable.

@sunce86
Copy link
Contributor Author

sunce86 commented Dec 27, 2023

Refactored, requesting review again.

@sunce86 sunce86 requested a review from fleupold December 27, 2023 15:36
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I like this approach much more. I still think we can factor the code inside fee.rs to make it easier to read and follow but it's mainly style preference.

I don't think the error handling is correct yet, and overflow handling for additions may be missing.

crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/mod.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me (just two small issues with static orders and multiple fee policies).

How do you want to go about adding tests? Are we going to keep the unit tests? In any case, I'd like to see a driver or integration test with the two examples we have in notion to make sure this works without coupling the test to the concrete code factoring.

crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/mod.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved

fn protocol_fee(&self, prices: ClearingPrices) -> Result<eth::U256, Error> {
let mut protocol_fee = eth::U256::zero();
for fee_policy in self.order().fee_policies.iter() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not an issue for now, but I wonder if the current logic would work for multiple fee policies (e.g. two 50% price improvements). I would think those would be applied sequentially (ie. we first get a fulfillment from applying the first fee - with less surplus! - and then only take 50% of the remaining surplus rather than 100%).

My suggestion would be to expose another apply method on a local impl extension of the FeePolicy enum, that takes a fulfilment and prices and returns a new fulfilment with its policy applied.

But we can also cross that bridge when we get to it, in this case we should probably document and guard with an error that multiple fee policies aren't yet properly implemented.

Copy link
Contributor Author

@sunce86 sunce86 Dec 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to think more if we should allow for multiple policies of the same kind. Does it make sense to have two price improvement policies on the same order, where the calculated fee amounts depend on the ordering of policies.
Would you expect two policies, one volume based, one price improvement, to also be sequentially applied?

@sunce86
Copy link
Contributor Author

sunce86 commented Dec 29, 2023

How do you want to go about adding tests? Are we going to keep the unit tests? In any case, I'd like to see a driver or integration test with the two examples we have in notion to make sure this works without coupling the test to the concrete code factoring.

I'm not going to keep the unit tests (they just show that the calculations are correct and match with the current surplus calculation we have, for example). They will exist in the PR history so we can reproduce them again in the future if needed.

I planned on adding e2e test when everything is done. Do you think it should be part of this PR? Considering that we send no policies at the moment, merging this PR is safe.

@sunce86 sunce86 requested review from fleupold and squadgazzz January 6, 2024 10:51
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙈 I don't think the current test setup lends itself nicely for this new feature and also needs some refactoring. My goal is that we can put the tests we have in Notion in code (and not come up with completely new examples that way fewer people have thought through and agreed upon)

crates/driver/src/tests/cases/protocol_fees.rs Outdated Show resolved Hide resolved
crates/driver/src/tests/setup/mod.rs Outdated Show resolved Hide resolved
crates/driver/src/tests/cases/protocol_fees.rs Outdated Show resolved Hide resolved
crates/driver/src/tests/cases/protocol_fees.rs Outdated Show resolved Hide resolved
crates/driver/src/tests/setup/blockchain.rs Outdated Show resolved Hide resolved
@sunce86 sunce86 mentioned this pull request Jan 10, 2024
@sunce86
Copy link
Contributor Author

sunce86 commented Jan 10, 2024

E2E tests are extracted into a separate PR:
#2260

@sunce86 sunce86 requested a review from fleupold January 10, 2024 13:54
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
Comment on lines 130 to 132
let surplus = limit_sell_amount
.checked_sub(executed_sell_amount_with_fee)
.unwrap_or(eth::U256::zero());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is used for the surplus based fee we need to reference the quoted price somehow.
Otherwise we'll only compare against the signed limit price which already has the slippage baked in. This is not what we want, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation only works for out of market limit orders (where the quote is worse than the limit price). It's true that the naming is a bit misleading and the real fee policy we are implementing here is is SurplusFee not PriceImprovement

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the PriceImprovement as a general term - improvement over SOME price, whether it's a limit price or some other artificial price (artificial because driver works with limit orders technically and doesn't care whether it has slippage baked in and in which amount).
If it were called MarketPriceImprovement then it would be misleading IMO. Having a general term is better since we will implement the improvement over quotes 99.99% so renaming it would lead to renaming the autopilot/driver interface which is not too problematic right now, but it could be in the future.
OTOH, I might be biased since I spent a lot of time thinking in those terms, so if you both prefer to have SurplusFee I will update it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've captured the potential renaming #2266
I believe this is ok since chances are higher that we will implement this so it's easier to close the issue than to revert the changes.

crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is true that it is quite inconsistent that we implement a "price improvement fee" but cannot actually take it from the price improvement (since we don't know the quotes).

I'm not sure if it's worthwhile renaming the FeePolicy type to Surplus for now which would be correct, since technically your slippage tolerance is part of the surplus.

Functionality wise it doesn't matter since it's only planned to apply to limit orders for now, but I wonder if we start creating a cognitive mess by introducing this inconsistency (which may or may not be cleaned up one day). In any case, there may be merit in having a general "surplus" fee policy on top of price improvement and volume.

Comment on lines 130 to 132
let surplus = limit_sell_amount
.checked_sub(executed_sell_amount_with_fee)
.unwrap_or(eth::U256::zero());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation only works for out of market limit orders (where the quote is worse than the limit price). It's true that the naming is a bit misleading and the real fee policy we are implementing here is is SurplusFee not PriceImprovement

crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/fee.rs Outdated Show resolved Hide resolved
crates/driver/src/infra/solver/dto/solution.rs Outdated Show resolved Hide resolved
crates/driver/src/domain/competition/solution/trade.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@MartinquaXD MartinquaXD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only nits remaining for me.
Approving since both sides of the renaming discussion make sense to me and I don't have strong feelings about it.
Just wanted to point it out in case this was not intentional since I remembered the plan differently. But if this is just an intermediate PR until we have all the data needed for actually taking fees over price improvement compared to quoted amounts I'm fine with it.

Copy link
Contributor

@fleupold fleupold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can merge this PR as is, but let's track renaming the policy as a follow up issue we should decide on and if we want to do it execute before launch.

# Description
Merges to #2213, not main.

Adds e2e tests for protocol fee calculation. 
- Sell order
- Sell order, but protocol fee capped by max volume
- Buy order
- Buy order, but protocol fee capped by max volume

Asserts that if `executed_surplus_fee = X` without protocol fees, then
`executed_surplus_fee = X + protocol_fee` with protocol fees.
@sunce86 sunce86 merged commit 04b18f6 into main Jan 11, 2024
8 checks passed
@sunce86 sunce86 deleted the protocol-fee-adjustment-poc branch January 11, 2024 19:26
@github-actions github-actions bot locked and limited conversation to collaborators Jan 11, 2024
@sunce86 sunce86 restored the protocol-fee-adjustment-poc branch February 9, 2024 13:35
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
E:4.2 Protocol Fee Implementation See https://github.com/cowprotocol/pm/issues/29 for details
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants