Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ON HOLD] Refactor order_execution table #2104

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions crates/autopilot/src/database/competition.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@ use {

#[derive(Clone, Debug)]
pub enum ExecutedFee {
/// Fee is calculated by the solver and known upfront (before the settlement
/// Fee is determined by user and known upfront (before the settlement
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this struct even needed in the co-located world? Can't all fees (fixed order fee as well at the surplus fee) be determined by parsing the settlement in the end? Would this simplify the code paths?

/// is finalized).
Solver(U256),
UserFee(U256),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it's not really UserDetermined, but rather OrderDetermined?

How about ExecutedFee:Signed vs ExecutedFee::Surplus ?

/// Fee is unknown before the settlement is finalized and is calculated in
/// the postprocessing. Currently only used for limit orders.
Surplus,
SolverDetermined,
}

impl ExecutedFee {
pub fn fee(&self) -> Option<&U256> {
match self {
ExecutedFee::Solver(fee) => Some(fee),
ExecutedFee::Surplus => None,
ExecutedFee::UserFee(fee) => Some(fee),
ExecutedFee::SolverDetermined => None,
}
}
}
Expand Down Expand Up @@ -82,13 +82,15 @@ impl super::Postgres {
.context("solver_competition::save")?;

for order_execution in &competition.order_executions {
let solver_fee = order_execution.executed_fee.fee().map(u256_to_big_decimal);
database::order_execution::save(
&mut ex,
&ByteArray(order_execution.order_id.0),
competition.auction_id,
None,
solver_fee.as_ref(),
order_execution
.executed_fee
.fee()
.map(u256_to_big_decimal)
.as_ref(),
)
.await
.context("order_execution::save")?;
Expand Down
11 changes: 4 additions & 7 deletions crates/autopilot/src/database/on_settlement_event_updater.rs
Original file line number Diff line number Diff line change
Expand Up @@ -113,15 +113,12 @@ impl super::Postgres {

if insert_succesful || matches!(auction_data.auction_id, AuctionId::Centralized(_)) {
// update order executions for orders with solver computed fees (limit orders)
// for limit orders, fee is called surplus_fee and is determined by the solver
// therefore, when transaction is settled onchain we calculate the fee and save
// it to DB
for order_execution in auction_data.order_executions {
database::order_execution::update_surplus_fee(
for (order_uid, executed_fee) in auction_data.order_executions {
database::order_execution::update_executed_fee(
ex,
&ByteArray(order_execution.0 .0), // order uid
&ByteArray(order_uid.0),
auction_data.auction_id.assume_verified(),
&u256_to_big_decimal(&order_execution.1), // order fee
&u256_to_big_decimal(&executed_fee),
)
.await
.context("insert_missing_order_executions")?;
Expand Down
43 changes: 16 additions & 27 deletions crates/autopilot/src/decoded_settlement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ use {
anyhow::{Context, Result},
bigdecimal::{Signed, Zero},
contracts::GPv2Settlement,
database::orders::OrderClass,
ethcontract::{common::FunctionExt, tokens::Tokenize, Address, Bytes, H160, U256},
model::{
order::{OrderKind, OrderUid},
Expand Down Expand Up @@ -138,15 +137,13 @@ impl From<(Address, U256, Bytes<Vec<u8>>)> for DecodedInteraction {
#[derive(Debug, Clone)]
pub struct OrderExecution {
pub order_uid: OrderUid,
pub executed_solver_fee: Option<U256>,
pub executed_fee: Option<U256>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need to be Option?

Copy link
Contributor

@MartinquaXD MartinquaXD Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like on_settlement_event_updater (which updates the required DB entries when a settlement gets indexed) requires 2 pieces of information:

  1. the full (signed) fee amount collected in total by the settlement
  2. all OrderExecutions that don't have a fee assigned to them yet

These pieces of information can currently be served by the same function Postgres::order_executions_for_tx() which returns all OrderExecutions (with and without a signed fee). For 1 we currently sum all the fees which are already populated and for 2 we use only the executions where executed_fee.is_none().

So refactoring this field to be non-optional might make other code weirder.

While looking into that I stumbled over this code. Could somebody give that a sanity check. My current understanding is that this code is correct because it should only contain the fees that were signed upfront by the user because those are the only fees that are part of the objective value computation. But because fees have become so complicated since we introduced fees that were not signed upfront I think checking that logic once more might be good.

pub sell_token: H160,
pub buy_token: H160,
pub sell_amount: U256,
pub buy_amount: U256,
pub executed_amount: U256,
pub signature: Vec<u8>, //encoded signature
// For limit orders the solver computes the fee
pub solver_determines_fee: bool,
pub signature: Vec<u8>, // encoded signature
}

impl TryFrom<database::orders::OrderExecution> for OrderExecution {
Expand All @@ -155,10 +152,7 @@ impl TryFrom<database::orders::OrderExecution> for OrderExecution {
fn try_from(order: database::orders::OrderExecution) -> std::result::Result<Self, Self::Error> {
Ok(Self {
order_uid: OrderUid(order.order_uid.0),
executed_solver_fee: order
.executed_solver_fee
.as_ref()
.and_then(big_decimal_to_u256),
executed_fee: order.executed_fee.as_ref().and_then(big_decimal_to_u256),
sell_token: H160(order.sell_token.0),
buy_token: H160(order.buy_token.0),
sell_amount: big_decimal_to_u256(&order.sell_amount).context("sell_amount")?,
Expand All @@ -171,7 +165,6 @@ impl TryFrom<database::orders::OrderExecution> for OrderExecution {
.encode_for_settlement(H160(order.owner.0))
.to_vec()
},
solver_determines_fee: order.class == OrderClass::Limit,
})
}
}
Expand Down Expand Up @@ -255,7 +248,7 @@ impl DecodedSettlement {
})
}

/// Returns the total `executed_solver_fee` of this solution converted to
/// Returns the total `executed_fee` of this solution converted to
/// the native token. This is only the value used for objective value
/// computatations and can theoretically be different from the value of
/// fees actually collected by the protocol.
Expand Down Expand Up @@ -312,10 +305,10 @@ impl DecodedSettlement {
// use every `OrderExecution` exactly once.
let order = orders.swap_remove(i);

// Update fee only for orders with solver computed fees (limit orders)
if !order.solver_determines_fee {
// Update fee for orders for which is not calculated yet.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Update fee for orders for which is not calculated yet.
// Update fee for orders for which it is not calculated yet.

if order.executed_fee.is_some() {
return None;
}
};

let fees = self.fee(external_prices, &order, trade);

Expand All @@ -333,8 +326,8 @@ impl DecodedSettlement {
order: &OrderExecution,
trade: &DecodedTrade,
) -> Option<Fees> {
let solver_fee = match &order.executed_solver_fee {
Some(solver_fee) => *solver_fee,
let executed_fee = match order.executed_fee {
Some(executed_fee) => executed_fee,
None => {
// get uniform prices
let sell_index = self
Expand Down Expand Up @@ -385,14 +378,14 @@ impl DecodedSettlement {

// converts the order's `solver_fee` which is denominated in `sell_token` to the
// native token.
tracing::trace!(?solver_fee, "fee before conversion to native token");
tracing::trace!(?executed_fee, "fee before conversion to native token");
let fee = external_prices
.try_get_native_amount(order.sell_token, u256_to_big_rational(&solver_fee))?;
.try_get_native_amount(order.sell_token, u256_to_big_rational(&executed_fee))?;
tracing::trace!(?fee, "fee after conversion to native token");

Some(Fees {
order: order.order_uid,
sell: solver_fee,
sell: executed_fee,
native: big_rational_to_u256(&fee).ok()?,
})
}
Expand Down Expand Up @@ -730,25 +723,23 @@ mod tests {
let orders = vec![
OrderExecution {
order_uid: OrderUid::from_str("0xa8b0c9be7320d1314c6412e6557efd062bb9f97f2f4187f8b513f50ff63597cae995e2a9ae5210feb6dd07618af28ec38b2d7ce163f4d8c4").unwrap(),
executed_solver_fee: Some(48263037u128.into()),
executed_fee: Some(48263037u128.into()),
buy_amount: 11446254517730382294118u128.into(),
sell_amount: 14955083027u128.into(),
sell_token: addr!("dac17f958d2ee523a2206206994597c13d831ec7"),
buy_token: Default::default(),
executed_amount: 14955083027u128.into(),
signature: hex::decode("155ff208365bbf30585f5b18fc92d766e46121a1963f903bb6f3f77e5d0eaefb27abc4831ce1f837fcb70e11d4e4d97474c677469240849d69e17f7173aead841b").unwrap(),
solver_determines_fee: false,
},
OrderExecution {
order_uid: OrderUid::from_str("0x82582487739d1331572710a9283dc244c134d323f309eb0aac6c842ff5227e90f352bffb3e902d78166a79c9878e138a65022e1163f4d8bb").unwrap(),
executed_solver_fee: Some(127253135942751092736u128.into()),
executed_fee: Some(127253135942751092736u128.into()),
buy_amount: 1236593080.into(),
sell_amount: 5701912712048588025933u128.into(),
sell_token: addr!("f4d2888d29d722226fafa5d9b24f9164c092421e"),
buy_token: Default::default(),
executed_amount: 5701912712048588025933u128.into(),
signature: hex::decode("882a1c875ff1316bb79bde0d0792869f784d58097d8489a722519e6417c577cf5cc745a2e353298dea6514036d5eb95563f8f7640e20ef0fd41b10ccbdfc87641b").unwrap(),
solver_determines_fee: false,
}
];
let fees = settlement
Expand Down Expand Up @@ -806,14 +797,13 @@ mod tests {
let orders = vec![
OrderExecution {
order_uid: OrderUid::from_str("0xaa6ff3f3f755e804eefc023967be5d7f8267674d4bae053eaca01be5801854bf6c7f534c81dfedf90c9e42effb410a44e4f8ef1064690e05").unwrap(),
executed_solver_fee: None,
executed_fee: None,
buy_amount: 11446254517730382294118u128.into(), // irrelevant
sell_amount: 14955083027u128.into(), // irrelevant
sell_token: addr!("ba386a4ca26b85fd057ab1ef86e3dc7bdeb5ce70"),
buy_token: addr!("c02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"),
executed_amount: 134069619089011499167823218927u128.into(),
signature: hex::decode("f8ad81db7333b891f88527d100a06f23ff4d7859c66ddd71514291379deb8ff660f4fb2a24173eaac5fad2a124823e968686e39467c7f3054c13c4b70980cc1a1c").unwrap(),
solver_determines_fee: true,
},
];
let fees = settlement
Expand Down Expand Up @@ -920,14 +910,13 @@ mod tests {
let orders = vec![
OrderExecution {
order_uid: Default::default(),
executed_solver_fee: Some(463182886014406361088u128.into()),
executed_fee: Some(463182886014406361088u128.into()),
buy_amount: 89238894792574185u128.into(),
sell_amount: 3026871740084629982950u128.into(),
sell_token: addr!("f88baf18fab7e330fa0c4f83949e23f52fececce"),
buy_token: addr!("eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"),
executed_amount: 0.into(),
signature: hex::decode("4935ea3f24155f6757df94d8c0bc96665d46da51e1a8e39d935967c9216a60912fa50a5393a323d453c78d179d0199ddd58f6d787781e4584357d3e0205a76001c").unwrap(),
solver_determines_fee: false,
},
];
let fees = settlement
Expand Down
6 changes: 3 additions & 3 deletions crates/autopilot/src/run_loop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -182,10 +182,10 @@ impl RunLoop {
match auction_order {
Some(auction_order) => {
let executed_fee = match auction_order.solver_determines_fee() {
// we don't know the surplus fee in advance. will be populated
// we don't know the executed fee in advance. will be populated
// after the transaction containing the order is mined
true => ExecutedFee::Surplus,
false => ExecutedFee::Solver(auction_order.metadata.solver_fee),
true => ExecutedFee::SolverDetermined,
false => ExecutedFee::UserFee(auction_order.metadata.solver_fee),
};
order_executions.push(OrderExecution {
order_id: *order_id,
Expand Down
32 changes: 11 additions & 21 deletions crates/database/src/order_execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,39 +9,37 @@ pub async fn save(
ex: &mut PgConnection,
order: &OrderUid,
auction: AuctionId,
surplus_fee: Option<&BigDecimal>,
solver_fee: Option<&BigDecimal>,
executed_fee: Option<&BigDecimal>,
) -> Result<(), sqlx::Error> {
const QUERY: &str = r#"
INSERT INTO order_execution (order_uid, auction_id, reward, surplus_fee, solver_fee)
INSERT INTO order_execution (order_uid, auction_id, reward, executed_fee)
VALUES ($1, $2, $3, $4, $5)
;"#;
sqlx::query(QUERY)
.bind(order)
.bind(auction)
.bind(0.) // reward is deprecated but saved for historical analysis
.bind(surplus_fee)
.bind(solver_fee)
.bind(executed_fee)
.execute(ex)
.await?;
Ok(())
}

// update already existing order_execution record with surplus_fee for partial
// update already existing order_execution record with executed_fee for partial
// limit orders
pub async fn update_surplus_fee(
pub async fn update_executed_fee(
mut ex: &mut PgConnection,
order: &OrderUid,
auction: AuctionId,
surplus_fee: &BigDecimal,
executed_fee: &BigDecimal,
) -> Result<(), sqlx::Error> {
const QUERY: &str = r#"
UPDATE order_execution
SET surplus_fee = $1
SET executed_fee = $1
WHERE order_uid = $2 AND auction_id = $3
;"#;
sqlx::query(QUERY)
.bind(surplus_fee)
.bind(executed_fee)
.bind(order)
.bind(auction)
.execute(ex.deref_mut())
Expand All @@ -60,18 +58,10 @@ mod tests {
let mut db = db.begin().await.unwrap();
crate::clear_DANGER_(&mut db).await.unwrap();

save(&mut db, &Default::default(), 0, None, Default::default())
save(&mut db, &Default::default(), 0, None).await.unwrap();

save(&mut db, &Default::default(), 1, Some(&Default::default()))
.await
.unwrap();

save(
&mut db,
&Default::default(),
1,
Some(&Default::default()),
Default::default(),
)
.await
.unwrap();
}
}
Loading
Loading