fix: catch and ignore provider errors during retry

[sanjayprabhu]

Motivation

Currently hub crashes when it hits a rate limit error:

hubble-hubble-1  | {"level":50,"time":1699646142590,"pid":1,"hostname":"58bbe902717e","reason":"Unhandled Rejection","err":{"type":"HttpRequestError","message":"HTTP request failed.\n\nStatus: 429\nURL: https://opt-mainnet.g.alchemy.com/v2/PRIVATE_KEY\nRequest body: {\"method\":\"eth_getFilterLogs\",\"params\":[\"0xd01c87ecce67458269476e637364f209\"]}\n\nDetails: {\"code\":429,\"message\":\"Your app has exceeded its compute units per second capacity. If you have retries enabled, you can safely ignore this message. If not, check out https://docs.alchemy.com/[email protected]","stack":"HttpRequestError: HTTP request failed.\n\nStatus: 429\nURL: https://opt-mainnet.g.alchemy.com/v2/PRIVATE_KEY\nRequest body: {\"method\":\"eth_getFilterLogs\",\"params\":[\"0xd01c87ecce67458269476e637364f209\"]}\n\nDetails: {\"code\":429,\"message\":\"Your app has exceeded its compute units per second capacity. If you have retries enabled, you can safely ignore this message. If not, check out https://docs.alchemy.com/[email protected]\n    at Object.http (file:///home/node/app/node_modules/viem/_esm/utils/rpc.js:42:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async fn (file:///home/node/app/node_modules/viem/_esm/clients/transports/http.js:39:24)\n    at async request (file:///home/node/app/node_modules/viem/_esm/clients/transports/http.js:40:45)\n    at async withRetry.delay.count.count (file:///home/node/app/node_modules/viem/_esm/utils/buildRequest.js:26:20)\n    at async attemptRetry (file:///home/node/app/node_modules/viem/_esm/utils/promise/withRetry.js:12:30)","details":"{\"code\":429,\"message\":\"Your app has exceeded its compute units per second capacity. If you have retries enabled, you can safely ignore this message. If not, check out https://docs.alchemy.com/refPRIVATE_KEYges":["Status: 429","URL: https://opt-mainnet.g.alchemy.com/v2/PRIVATE_KEY","Request body: {\"method\":\"eth_getFilterLogs\",\"params\":[\"0xd01c87ecce67458269476e637364f209\"]}"],"shortMessage":"HTTP request failed.","name":"HttpRequestError","version":"[email protected]","body":{"method":"eth_getFilterLogs","params":["0xd01c87ecce67458269476e637364f209"]},"headers":{},"status":429,"url":"https://opt-mainnet.g.alchemy.com/v2/PRIVATE_KEY"},"msg":"shutting down hub"}

Change Summary

Describe the changes being made in 1-2 concise sentences.

Merge Checklist

Choose all relevant options below by adding an x now or at any time before submitting for review

• PR title adheres to the conventional commits standard
• PR has a changeset
• PR has been tagged with a change label(s) (i.e. documentation, feature, bugfix, or chore)
• PR includes documentation if necessary.
• All commits have been signed

Additional Context

If this is a relatively large or complex change, provide more details here that will help reviewers

---

PR-Codex overview

Focus of the PR:

This PR focuses on fixing an issue where provider errors during retry were not being caught and ignored.

Detailed summary:

• Added a try-catch block to catch and handle errors during the retry of historical events synchronization.
• If an error occurs during the retry, it is logged with the corresponding block number.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

[changeset-bot]

🦋 Changeset detected

Latest commit: 6a48513

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@farcaster/hubble	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
hub-monorepo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 10, 2023 11:10pm

[codecov]

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Files	Coverage Δ
apps/hubble/src/eth/l2EventsProvider.ts	68.43% <0.00%> (-0.49%)	⬇️

... and 1 file with indirect coverage changes

📢 Thoughts on this report? Let us know!