Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement queries throttle for switch account component #5062

Merged
merged 1 commit into from
Mar 25, 2024

Conversation

alter-eggo
Copy link
Contributor

@alter-eggo alter-eggo commented Mar 12, 2024

Try out Leather build 6c5eb45Extension build, Test report, Storybook, Chromatic

This pr throttles all balances queries on switch account list component, to prevent 429 errors

@alter-eggo alter-eggo force-pushed the feat/debounced-query branch from b54a852 to c42e1d2 Compare March 13, 2024 14:03
@alter-eggo alter-eggo linked an issue Mar 13, 2024 that may be closed by this pull request
@alter-eggo alter-eggo changed the title feat: implement inscriptions debounced query feat: implement queries throttle for switch account component Mar 13, 2024
Copy link
Collaborator

@kyranjamie kyranjamie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding the debounce directly to the hooks breaks the SRP, and generally makes them more complicated than they should be. Perhaps there might be better ways to handle adding a delay before requesting directly in the hook?

The main scenario I think about is the switch accounts lists. If a user scrolls down 300 accounts, we shouldn't queue up 300x balance and 300x bns name requests.

I reckon there are two mechanisms we could alternatively use to ensure that requests that aren't needed, don't get triggered

  1. Client side rate limiting
    We use https://github.com/jhurliman/node-rate-limiter to ensure that we don't fire too many requests at once. These could be configured to trigger substantially less than the actual RPM, just to ensure we don't use too much. If a request is triggered and added to the queue then the component dismounts before the request is made, we use the AbortController to cancel the promise (and could even add back one unspent request to the rate limtier)
  2. Component tree debounce.
    Another way to delay the request could be internal to the account item component. We could write code where we only render the component that has the query hook, 1second after render. This way, the request is never made until the component mounts. This is the same principal as what you've built here, but implemented in a way where we don't change the query hook implementatoin.

src/shared/utils/query.ts Outdated Show resolved Hide resolved
@alter-eggo alter-eggo force-pushed the feat/debounced-query branch from c42e1d2 to 4fce13e Compare March 13, 2024 16:51
@alter-eggo alter-eggo marked this pull request as ready for review March 13, 2024 17:15
@alter-eggo alter-eggo force-pushed the feat/debounced-query branch 2 times, most recently from cd1142d to b01dd24 Compare March 13, 2024 19:16
Copy link
Collaborator

@kyranjamie kyranjamie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I have a request queued by the rate limiter, and the component dismounts, what happens?

src/app/query/stacks/rate-limiter.ts Outdated Show resolved Hide resolved
src/app/query/bitcoin/blockstream-rate-limiter.ts Outdated Show resolved Hide resolved
@markmhendrickson
Copy link
Collaborator

@314159265359879 want to QA this one by trying to "break" it with wallets that have lots of accounts and assets?

@314159265359879
Copy link
Contributor

314159265359879 commented Mar 14, 2024

tested this version
image

I see a lot of these errors (coingecko api) as I scroll through accounts to select the right one (testing with 400 accounts wallet):
image is it checking the USD price per account? use old value unless more than 5 minutes old instead?

As soon as I open "select account" screen I get these 429's from hiro api's
image
looks like some bnsx calls and ordinals inscription calls.

Another odd thing, I am not sure if it is related to this PR when I switch from an account with a name to an account without a name the name remains displayed instead of "Account x"
image

I see a lot less 429's as I scroll over the accounts. I am still waiting for names to load though and there is no indication the wallet is doing anything. I feel like some of the API budget should be reserved so that BNS names and balances can be checked once the user stops scrolling for a few seconds. Or a way to force where to update next.

Scenario
I feel like I am close to the account I need (Account 160) but it could also be 161 or 162... once the name is loaded I would know.
Clicking the account... nothing happens because it is still waiting before making additional calls. Displaying the wrong name. I feel like it should have api budget to check that name right away or display the (correct) account number instead.

Could the wallet check for BNS names for all accounts (in the background even before account selection is started) and just save them to persist? And only update those when the user asks/click refresh-name? Names are not expected to change that often but they are rather vital for identifying accounts.
If loading the balances prevents it from loading data when an account is selected/entered into I don't think we're helping the user yet.
What about not loading any balances automatically on the account selection screen unless the user asks for it, a "load now" button" next to each account or a global one that starts loading balances from the top of what is visible when it is clicked?

Perhaps some of that is out of scope for this PR.
Leaving some API budget to check BNS name when account selected seems vital though.
I wonder if loading should pause for a bit longer before starting so the user has a chance to move to where they want the loading to start. I am not sure if the wait time could be made dependent on how many accounts are revealed in the wallet. I am sure you can start loading right away when there are only 5 accounts. When there are a hundred, let them roughly target first?

@314159265359879
Copy link
Contributor

it looks like the coingecko API issue (error 403) was related to IP restrictions, please ignore. I don't see it now that I switched to another IP.

@alter-eggo alter-eggo force-pushed the feat/debounced-query branch 4 times, most recently from bf0b234 to 6d18d64 Compare March 14, 2024 19:22
@alter-eggo
Copy link
Contributor Author

alter-eggo commented Mar 14, 2024

@314159265359879 wonder if you could test latest build please

@314159265359879
Copy link
Contributor

I have waited until it does something again but it is not loading the name yet after selecting. I think it is prioritizing loading bitcoin data perhaps... there is no bitcoin in any of these accounts so it is a bit odd.
image

When I switch account scroll down from 156 to 400... I see one name loading. The rest of these accounts have names too but they are not yet displayed.
image

is it possible to prioritize loading the names?

@alter-eggo alter-eggo force-pushed the feat/debounced-query branch from d5e474c to 675955a Compare March 15, 2024 11:54
@alter-eggo alter-eggo force-pushed the feat/debounced-query branch 2 times, most recently from d1fb2ff to d2c0b7a Compare March 20, 2024 11:33
@alter-eggo alter-eggo requested a review from kyranjamie March 20, 2024 11:33
@alter-eggo alter-eggo force-pushed the feat/debounced-query branch 8 times, most recently from 31d9b12 to 5c07103 Compare March 22, 2024 16:57
@@ -2,11 +2,21 @@ import { memo } from 'react';

import { styled } from 'leather-styles/jsx';

import { shimmerStyles } from '../../../../theme/global/shimmer-styles';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some pt we should add an alias for theme folder.

@fbwoolf
Copy link
Contributor

fbwoolf commented Mar 22, 2024

@alter-eggo I downloaded the latest build here ...none of my Stacks NFTs are loading. 🤔

@alter-eggo alter-eggo force-pushed the feat/debounced-query branch from 5c07103 to 6ab31f4 Compare March 23, 2024 13:32
@alter-eggo
Copy link
Contributor Author

@fbwoolf thanks for testing! fixed

Copy link
Collaborator

@kyranjamie kyranjamie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff @alter-eggo, just tried this out and works very well. Maybe we'll need to fine-tune some of the priority settings later, but this should already be such a huge improvement. Look forward to checking 429 errors pre/post these changes.

@alter-eggo alter-eggo force-pushed the feat/debounced-query branch from 6ab31f4 to 6c5eb45 Compare March 24, 2024 17:53
@alter-eggo alter-eggo added this pull request to the merge queue Mar 25, 2024
Merged via the queue into dev with commit b1b2ec5 Mar 25, 2024
30 checks passed
@alter-eggo alter-eggo deleted the feat/debounced-query branch March 25, 2024 09:28
@markmhendrickson
Copy link
Collaborator

Does this apply to "Select account" for the recipient field of the send form as well?

@alter-eggo
Copy link
Contributor Author

@markmhendrickson yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Throttle queries when viewing many accounts for selection
5 participants