Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add RPC requests logging for RETH calls slower than GETH #785

Conversation

jsy1218
Copy link
Member

@jsy1218 jsy1218 commented Jul 18, 2024

I observe that RETH RPC calls are slower than GETH calls in Uniswap routing backend. My observation is based on two different time periods:

  1. Between may 20th and may 21th, I had a PR that only samples quicknode reth and another PR that fail over from infura to quicknode geth. This means, before May 20th, I had the shadow sampling of mix of geth and reth, meanwhile between may 20th and may 21th, I had the shadow sampling of only reth.

request volume dropped in half on may 20th, because I changed from shadow sampling both quicknode geth and reth to shadow sampling only quicknode reth:

Screenshot 2024-07-18 at 8.00.52 PM.png

If reth were to run faster than geth, then I expect P50 latency to go down, but instead it went up:

Screenshot 2024-07-18 at 8.01.09 PM.png

  1. Recently I created a PR to split geth vs reth latency metrics.

Sampling traffic volume are comparable between geth and reth, just to make sure the benchmarking is on the same traffic scale:

Screenshot 2024-07-18 at 8.04.21 PM.png

I can see that P50 is also higher on reth than geth:

Screenshot 2024-07-18 at 8.05.36 PM.png

I do believe this is due to Uniswap specific RPC call pattern, since we make heavy multicall for quoting. e.g. this tenderly simulation captures a heavy multicall with hundreds of routes. As mentioned in #775, from Paradigm flood testing, we see quicknode reth is faster than quicknode geth, so this makes me believe it's uniswap specific RPC call pattern.

I think it makes sense to log the RPC requests when we observe the evaluated latencies are slower than the current provider latencies. This can help further debug why RETH for Uniswap is slower.

Copy link
Member Author

jsy1218 commented Jul 18, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jsy1218 and the rest of your teammates on Graphite Graphite

@jsy1218 jsy1218 marked this pull request as ready for review July 18, 2024 18:08
@graphite-app graphite-app bot requested review from xrsv, uni-guillaume, cgkol and a team July 18, 2024 18:14
@graphite-app graphite-app bot requested a review from mikeki July 18, 2024 18:14
Copy link

graphite-app bot commented Jul 18, 2024

Graphite Automations

"Request reviewers once CI passes on routing-api repo" took an action on this PR • (07/18/24)

6 reviewers were added and 1 assignee was added to this PR based on 's automation.

@jsy1218
Copy link
Member Author

jsy1218 commented Jul 18, 2024

I can see the new logging showed up in my local AWS routing-api:
Screenshot 2024-07-18 at 8 21 40 PM

Copy link
Member Author

jsy1218 commented Jul 18, 2024

Merge activity

  • Jul 18, 4:11 PM EDT: @jsy1218 started a stack merge that includes this pull request via Graphite.
  • Jul 18, 4:11 PM EDT: @jsy1218 merged this pull request with Graphite.

@jsy1218 jsy1218 merged commit e0fc934 into main Jul 18, 2024
5 checks passed
@jsy1218 jsy1218 deleted the siyujiang/route-198-log-rpc-requests-for-reth-calls-that-are-slower-than-geth branch July 18, 2024 20:11
mikeki added a commit that referenced this pull request Jul 25, 2024
mikeki added a commit that referenced this pull request Jul 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants