Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: improve transactions and messages storing #2

Merged
merged 11 commits into from
Apr 4, 2024

Conversation

RiccardoM
Copy link
Collaborator

@RiccardoM RiccardoM commented Apr 4, 2024

Description

This PR completely changes how transactions and messages are stored inside the database.

All tables have changed their name to follow the table naming convention of using plurals instead of singular names:

Old Name New Name
validator validators
block blocks
pre_commit pre_commits
transaction transactions
message messages

Also, the transactions and messages tables scheme have changed. Here are the new ones:

CREATE TABLE transactions
(
    hash         TEXT    NOT NULL,
    height       BIGINT  NOT NULL REFERENCES blocks (height),
    success      BOOLEAN NOT NULL,

    -- memo and signatures are part of the transaction body
    memo         TEXT,
    signatures   TEXT[]  NOT NULL,

    -- signer_infos and fee are part of the transaction auth info
    signer_infos JSONB   NOT NULL DEFAULT '[]'::JSONB,
    fee          JSONB   NOT NULL DEFAULT '{}'::JSONB,

    -- Additional fields from the transaction response
    gas_wanted   BIGINT           DEFAULT 0,
    gas_used     BIGINT           DEFAULT 0,
    raw_log      TEXT,
    logs         JSONB,

    -- partition_id is used to partition the table and make queries faster
    partition_id BIGINT  NOT NULL DEFAULT 0,

    CONSTRAINT unique_tx UNIQUE (hash, partition_id)
) PARTITION BY LIST (partition_id);

CREATE TABLE messages
(
    index            BIGINT NOT NULL,
    type             TEXT   NOT NULL,
    value            JSONB  NOT NULL,

    -- reference to the transaction table
    transaction_hash TEXT   NOT NULL,
    partition_id     BIGINT NOT NULL DEFAULT 0,
    FOREIGN KEY (transaction_hash, partition_id) REFERENCES transactions (hash, partition_id),

    CONSTRAINT unique_message_per_tx UNIQUE (transaction_hash, index, partition_id)
) PARTITION BY LIST (partition_id);

Finally, a new table named message_involved_addresses has been introduced:

CREATE TABLE message_involved_accounts
(
    user_address     TEXT   NOT NULL,

    -- reference to the messages table
    message_index    BIGINT NOT NULL,
    transaction_hash TEXT   NOT NULL,
    partition_id     BIGINT NOT NULL DEFAULT 0,
    FOREIGN KEY (transaction_hash, message_index, partition_id) REFERENCES messages (transaction_hash, index, partition_id),

    CONSTRAINT unique_message_account UNIQUE (message_index, transaction_hash, user_address, partition_id)
) PARTITION BY LIST (partition_id);

The module messages has been removed, as messages are now stored inside the database when storing a transaction (without the need for an external module doing that).

Two new methods have been added to the parsecmd.Config type: WithTransactionFilter and WithMessageFilterBuilder. These allow to specify specific filtering rules that will be used to determine whether a transaction or message should be stored or not inside the database.

Finally, the message.type field now is populated usind the messagbe type URL generated by the Cosmos SDK, instead of the generic Proto name generated by gogoproto.

Note

Due to the high breaking changes that this commit brings, we suggest dropping the database tables and migrating existing data using custom scripts. We do not provide a migration procedure for this update.

Checklist

  • Targeted PR against correct branch.
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Wrote unit tests.
  • Re-reviewed Files changed in the Github PR explorer.

@RiccardoM RiccardoM merged commit 12755db into main Apr 4, 2024
3 checks passed
@RiccardoM RiccardoM deleted the riccardom/refactor branch April 4, 2024 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant