Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Cloudflare Durable Objects #31

Open
whoabuddy opened this issue Apr 30, 2023 · 1 comment
Open

Implement Cloudflare Durable Objects #31

whoabuddy opened this issue Apr 30, 2023 · 1 comment

Comments

@whoabuddy
Copy link
Contributor

Problem

The inscribe.news API will either return a value from KV, or,
fetch the data from Hiro, store it in KV, then return the value.

Each request to the API calls the async function getInscription(), which may be causing a race condition where two simultaneous requests are stepping on each others promises (this post has a good visual and description).

  • Request A creates a Promise
  • Request B accidentally awaits Request A's existing Promise
  • This creates an exception that doesn't show in the logs

This has happened in the past when multiple indexer instances were running at once, and it used to at least show "This request will never return a response" in the logs, which was not present this time.

The odd state is consistent though, no clear indicates and requests time out with 524 errors after about 1-2 minutes, see #30 as the most recent example.

Solution

This could be mitigated by converting the getInscription() helper function to a durable object class, which gives additional functionality:

  • guarantees uniqueness of an object, no leaked ops like above
  • can access and return KV value if found
  • can fetch from API and write KV data (consistently, too!)
  • supports websocket endpoints, as a client or server
  • supports alarms, which replace cron triggers
@whoabuddy
Copy link
Contributor Author

Possible Architecture

DO class that returns from KV or writes inscriptions
(replaces current getInscription() functionality)

  • invoked via API calls
  • returns KV data if found
  • handles writes for one-off requests that aren't found
  • returns data

DO class that indexes all known inscriptions

  • gets total from Hiro API, compares against
  • tracks missing inscriptions in KV, fetches and writes them
  • runs on a frequent alarm

The first one could be split into more pieces as well if we run into any errors when scaling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant