Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix (replicator): Backfill Reactions and Storage Tables #1589

Closed
wants to merge 8 commits into from
Closed

fix (replicator): Backfill Reactions and Storage Tables #1589

wants to merge 8 commits into from

Conversation

zachterrell57
Copy link
Contributor

@zachterrell57 zachterrell57 commented Nov 26, 2023

Motivation

In its current implementation the replicator does not backfill the storage_allocations or reactions table. This PR adds logic to do that

Change Summary

  • Add BackfillFidStorageAllocations job
  • Remove the storage event type from BackfillFidOtherOnChainEvents as it's now getting added to the chain_events table through processOnChainEvents
  • Add BackfillFidReactions to the BackfillFidData job

Merge Checklist

Choose all relevant options below by adding an x now or at any time before submitting for review

Additional Context

Adding these two backfill jobs increases the backfill time significantly. We should add the ability to bootstrap the replicator from a snapshot like hubble


PR-Codex overview

This PR focuses on backfilling various data for a given fid.

Detailed summary

  • backfillFidOtherOnChainEvents.ts:
    • Removed OnChainEventType.EVENT_TYPE_STORAGE_RENT from eventTypes array.
    • Added idRegisterEventTypes array.
  • backfillFidStorageAllocations.ts:
    • Added imports for OnChainEventType, getOnChainEventsByFidInBatchesOf, registerJob, and processOnChainEvents.
    • Exported BackfillFidStorageAllocations as a registered job.
    • Added a run method that backfills storage allocations for a given fid.
  • backfillFidData.ts:
    • Imported BackfillFidReactions and BackfillFidStorageAllocations.
    • Added backfilling for reactions and storage allocations for a given fid.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

Copy link

changeset-bot bot commented Nov 26, 2023

⚠️ No Changeset found

Latest commit: d14d8c7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

vercel bot commented Nov 26, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hub-monorepo ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 26, 2023 9:42pm

Copy link

codecov bot commented Nov 26, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f026ad1) 73.34% compared to head (d14d8c7) 73.27%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1589      +/-   ##
==========================================
- Coverage   73.34%   73.27%   -0.07%     
==========================================
  Files          92       92              
  Lines        8636     8636              
  Branches     1946     1946              
==========================================
- Hits         6334     6328       -6     
- Misses       2184     2190       +6     
  Partials      118      118              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zachterrell57 zachterrell57 marked this pull request as ready for review November 27, 2023 12:43
@sds
Copy link
Member

sds commented Nov 28, 2023

Thank you!

@sds
Copy link
Member

sds commented Nov 28, 2023

Merged your change with an additional modification in #1592.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants