indexer-alt: sum_coin_balances pipeline #20070
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Skipped Deployments
|
amnn
changed the title
amnn
force-pushed
the
amnn/idx-sum-coin-balances
branch
from
d7f7b19 to
3885b1d
Compare
amnn
force-pushed
the
amnn/idx-sum-coin-balances
branch
from
3885b1d to
7abbac0
Compare
## Description The `sum_coin_balances` pipeline also needs to represent object updates, so this change factors out a type that can represent an update to an object (at a particular version), or a deletion at that version. ## Test plan ``` sui$ cargo nextest run -p sui-indexer-alt ```
amnn
force-pushed
the
amnn/idx-sum-coin-balances
branch
from
7abbac0 to
7a9525d
Compare
| -- | ||
| -- This can be used to paginate the coin balances of a given address at an | ||
| -- instant in time, returning coins in descending balance order. | ||
| CREATE TABLE IF NOT EXISTS sum_coin_balances |
There was a problem hiding this comment.
naive q, why sum_ instead of just coin_balances?
There was a problem hiding this comment.
sum stands for "summary", it's the equivalent of the "_snapshot" from "objects_snapshot". There will be an additional "wal_coin_balances" that is like the old "objects_history".
|
|
||
| /// Each insert or update will include at most this many rows -- the size is chosen to maximize the | ||
| /// rows without hitting the limit on bind parameters. | ||
| const UPDATE_CHUNK_ROWS: usize = i16::MAX as usize / 5; |
There was a problem hiding this comment.
ditto, good to read 5 from schema
There was a problem hiding this comment.
We can go back over and provide some automation to calculate the number of fields based on model types, using a procedural macro:
https://linear.app/mysten-labs/issue/DVX-447/automatically-calculate-field-count
|
|
||
| // Deleted and wrapped coins | ||
| for change in tx.effects.object_changes() { | ||
| // The object is not deleted/wrapped, or if it is it was unwrapped in the same |
There was a problem hiding this comment.
I think the current comment is grammatically correct:
The object is not deleted/wrapped, or if it is (deleted or wrapped) it was unwrapped in the same transaction
| let serialized = bcs::to_bytes(&coin_type) | ||
| .map_err(|_| anyhow!("Failed to serialize type for {}", object.id()))?; | ||
|
|
||
| coin_types.insert(object.id(), serialized); |
There was a problem hiding this comment.
assumption check, does coin_type never change?
There was a problem hiding this comment.
An object's type cannot currently change, but we aren't using that fact here, right? The reason why we include the coin type here is to make filtering by the coin type efficient.
There was a problem hiding this comment.
if the type can change, one object ID is likely mapped to multiple types on different versions, then we will get type for each (id, version) instead of id only to be correct?
There was a problem hiding this comment.
This table is keyed by id, and it only tracks the latest version of each object, so it's not going to have multiple versions of any given object.
## Description Similar to `sum_obj_types` in that it is tracking the live object set, but this index only covers coin objects owned by addresses, and it orders them by balance, which allows queries to return them in decreasing balance order. ## Test plan Manually run the indexer on the first 100,000 checkpoints: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 100000 ```
amnn
force-pushed
the
amnn/idx-sum-coin-balances
branch
from
7a9525d to
6f3c2c5
Compare
Description
Similar to
sum_obj_typesin that it is tracking the live object set,but this index only covers coin objects owned by addresses, and it
orders them by balance, which allows queries to return them in
decreasing balance order.
Test plan
Manually run the indexer on the first 100,000 checkpoints:
Stack
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.