Skip to content

Commit

Permalink
feat: Multichain indexing (#42)
Browse files Browse the repository at this point in the history
  • Loading branch information
@brunomenezes
brunomenezes authored Aug 29, 2024
1 parent b25af50 commit e215b72
Show file tree
Hide file tree
Showing 28 changed files with 3,352 additions and 20,509 deletions.
13 changes: 4 additions & 9 deletions .env
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,11 @@
DB_NAME=squid
DB_PORT=23798
GQL_PORT=4350
# Check the README file for more information about the available environment variables.
CHAIN_IDS="1, 11155111"
# JSON-RPC node endpoint, both wss and https endpoints are accepted
CHAIN_ID="11155111"
# possible configurations
# RPC_URL_1=
# RPC_URL_11155111=
# RPC_URL_31337=
# RPC_URL_10=
# RPC_URL_11155420=
# RPC_URL_8453=
# RPC_URL_84532=
# RPC_RATE_LIMIT_1=
# NORMALLY FOR USE WITH LOCAL/DEV
# GENESIS_BLOCK_31337=22
# BLOCK_CONFIRMATIONS_31337=1
# BLOCK_CONFIRMATIONS_31337=0
46 changes: 36 additions & 10 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,45 @@
## Rollups Explorer Backend

[![Coverage Status](https://coveralls.io/repos/github/cartesi/rollups-explorer-api/badge.svg?branch=main)](https://coveralls.io/github/cartesi/rollups-explorer-api?branch=main)

Experimentation with subsquid as a new indexing protocol.
This is a Subsquid Indexing protocol API for Cartesi Rollups Contracts and Applications.

That project requires [docker](https://docker.com) to be installed so the backend can be used locally.

### Project structure

the following folders belongs to the `backend`

- abi/
- assets/
- db/
- lib/ (generated when building)
- src/
### Configuration (Environment Variables)

> Supported Chains: 1, 11155111, 10, 11155420, 8453, 84532, 31337
| Variables | Default | Description |
| :-------------------------: | :--------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| `CHAIN_IDS` | `11155111` | Comma separated supported chain ids to be indexed |
| `RPC_URL_1` | `https://rpc.ankr.com/eth` | Endpoint for Mainnet RPC node |
| `RPC_URL_11155111` | `https://rpc.ankr.com/eth_sepolia` | Endpoint for Sepolia RPC node |
| `RPC_URL_31337` | `http://127.0.0.1:8545` | Endpoint for local node |
| `RPC_URL_10` | `https://mainnet.optimism.io` | Endpoint for Optimism Mainnet RPC node |
| `RPC_URL_11155420` | `https://sepolia.optimism.io` | Endpoint for Optimism Sepolia RPC node |
| `RPC_URL_8453` | `https://mainnet.base.org` | Endpoint for Base Mainnet RPC node |
| `RPC_URL_84532` | `https://sepolia.base.org` | Endpoint for Base Sepolia RPC node |
| `RPC_RATE_LIMIT_{CHAIN_ID}` | `undefined` | Option to fine tune concurrent requests by rate limiting the requests to RPC node providers. Replace {CHAIN_ID} with a supported chain id and set a Number e.g. 15. That is good to avoid [429](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429) status code when using free tiers. The default is Full speed |

For use with local devnet for fine tunning and deal with any unexpected changes e.g. Foundry nightly versions.

| Variables | Default | Description |
| :-------------------------: | :-----: | :----------------------------------------------------------------------------------: |
| `GENESIS_BLOCK_31337` | `22` | Genesis block for indexer to start |
| `BLOCK_CONFIRMATIONS_31337` | `1` | Distance from the head block behind which all blocks are considered to be finalized. |

Subsquid Related variables on Development mode. Check the docker-compose and commands.json files.

| Variables | Default | Description |
| :--------: | :---------: | :------------------------: |
| `DB_NAME` | `squid` | Database name |
| `DB_PORT` | `5432` | Database port to be used |
| `DB_SSL` | `true` | Config for SSL connections |
| `DB_PASS` | `postgres` | Database password |
| `DB_HOST` | `localhost` | Database host |
| `DB_USER` | `postgres` | Database user |
| `GQL_PORT` | `4350` | GraphQL API port |

### Development

Expand Down
85 changes: 85 additions & 0 deletions db/migrations/1724312923354-Data.js
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
module.exports = class Data1724312923354 {
name = 'Data1724312923354'

async up(db) {
await db.query(`CREATE TABLE "chain" ("id" character varying NOT NULL, CONSTRAINT "PK_8e273aafae283b886672c952ecd" PRIMARY KEY ("id"))`)
await db.query(`ALTER TABLE "token" ADD "address" text NOT NULL`)
await db.query(`ALTER TABLE "token" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "erc20_deposit" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "nft" ADD "address" text NOT NULL`)
await db.query(`ALTER TABLE "nft" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "erc721_deposit" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "multi_token" ADD "address" text NOT NULL`)
await db.query(`ALTER TABLE "multi_token" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "erc1155_deposit" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "input" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "application" ADD "address" text NOT NULL`)
await db.query(`ALTER TABLE "application" ADD "chain_id" character varying`)
await db.query(`ALTER TABLE "application_factory" ADD "address" text NOT NULL`)
await db.query(`ALTER TABLE "application_factory" ADD "chain_id" character varying`)
await db.query(`CREATE INDEX "IDX_de7e0d7203642a7985ad1e5bf9" ON "token" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_98e0e62905bf3179bdbf443735" ON "token" ("address", "chain_id") `)
await db.query(`CREATE INDEX "IDX_8f6b794ba2a2b727b1929bb557" ON "erc20_deposit" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_1bcdd2b8e813324a2804544eb1" ON "nft" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_bfc419f78ddfbf14a8daef4abd" ON "nft" ("address", "chain_id") `)
await db.query(`CREATE INDEX "IDX_01a73626515aef160a3049a788" ON "erc721_deposit" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_a0b30e82e31660302a4af8cbe2" ON "multi_token" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_3d12b347a7a916b6975b8b6737" ON "multi_token" ("address", "chain_id") `)
await db.query(`CREATE INDEX "IDX_732c4624c6d3a7de141edf6f73" ON "erc1155_deposit" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_cf5fb31b5cf6719dc28adbb7d1" ON "input" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_29002c32f1d054e8f0ec4ea9cd" ON "input" ("id", "chain_id") `)
await db.query(`CREATE INDEX "IDX_13e23630dd6edeb8dd7062483f" ON "application" ("chain_id") `)
await db.query(`CREATE INDEX "IDX_6b4911d2e20363e9d2b4b24178" ON "application" ("address", "chain_id") `)
await db.query(`CREATE INDEX "IDX_402986e1692e825c3c773a56d6" ON "application_factory" ("chain_id") `)
await db.query(`ALTER TABLE "token" ADD CONSTRAINT "FK_de7e0d7203642a7985ad1e5bf97" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "erc20_deposit" ADD CONSTRAINT "FK_8f6b794ba2a2b727b1929bb5571" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "nft" ADD CONSTRAINT "FK_1bcdd2b8e813324a2804544eb1b" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "erc721_deposit" ADD CONSTRAINT "FK_01a73626515aef160a3049a7884" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "multi_token" ADD CONSTRAINT "FK_a0b30e82e31660302a4af8cbe2f" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "erc1155_deposit" ADD CONSTRAINT "FK_732c4624c6d3a7de141edf6f732" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "input" ADD CONSTRAINT "FK_cf5fb31b5cf6719dc28adbb7d1b" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "application" ADD CONSTRAINT "FK_13e23630dd6edeb8dd7062483f5" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
await db.query(`ALTER TABLE "application_factory" ADD CONSTRAINT "FK_402986e1692e825c3c773a56d62" FOREIGN KEY ("chain_id") REFERENCES "chain"("id") ON DELETE NO ACTION ON UPDATE NO ACTION`)
}

async down(db) {
await db.query(`DROP TABLE "chain"`)
await db.query(`ALTER TABLE "token" DROP COLUMN "address"`)
await db.query(`ALTER TABLE "token" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "erc20_deposit" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "nft" DROP COLUMN "address"`)
await db.query(`ALTER TABLE "nft" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "erc721_deposit" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "multi_token" DROP COLUMN "address"`)
await db.query(`ALTER TABLE "multi_token" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "erc1155_deposit" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "input" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "application" DROP COLUMN "address"`)
await db.query(`ALTER TABLE "application" DROP COLUMN "chain_id"`)
await db.query(`ALTER TABLE "application_factory" DROP COLUMN "address"`)
await db.query(`ALTER TABLE "application_factory" DROP COLUMN "chain_id"`)
await db.query(`DROP INDEX "public"."IDX_de7e0d7203642a7985ad1e5bf9"`)
await db.query(`DROP INDEX "public"."IDX_98e0e62905bf3179bdbf443735"`)
await db.query(`DROP INDEX "public"."IDX_8f6b794ba2a2b727b1929bb557"`)
await db.query(`DROP INDEX "public"."IDX_1bcdd2b8e813324a2804544eb1"`)
await db.query(`DROP INDEX "public"."IDX_bfc419f78ddfbf14a8daef4abd"`)
await db.query(`DROP INDEX "public"."IDX_01a73626515aef160a3049a788"`)
await db.query(`DROP INDEX "public"."IDX_a0b30e82e31660302a4af8cbe2"`)
await db.query(`DROP INDEX "public"."IDX_3d12b347a7a916b6975b8b6737"`)
await db.query(`DROP INDEX "public"."IDX_732c4624c6d3a7de141edf6f73"`)
await db.query(`DROP INDEX "public"."IDX_cf5fb31b5cf6719dc28adbb7d1"`)
await db.query(`DROP INDEX "public"."IDX_29002c32f1d054e8f0ec4ea9cd"`)
await db.query(`DROP INDEX "public"."IDX_13e23630dd6edeb8dd7062483f"`)
await db.query(`DROP INDEX "public"."IDX_6b4911d2e20363e9d2b4b24178"`)
await db.query(`DROP INDEX "public"."IDX_402986e1692e825c3c773a56d6"`)
await db.query(`ALTER TABLE "token" DROP CONSTRAINT "FK_de7e0d7203642a7985ad1e5bf97"`)
await db.query(`ALTER TABLE "erc20_deposit" DROP CONSTRAINT "FK_8f6b794ba2a2b727b1929bb5571"`)
await db.query(`ALTER TABLE "nft" DROP CONSTRAINT "FK_1bcdd2b8e813324a2804544eb1b"`)
await db.query(`ALTER TABLE "erc721_deposit" DROP CONSTRAINT "FK_01a73626515aef160a3049a7884"`)
await db.query(`ALTER TABLE "multi_token" DROP CONSTRAINT "FK_a0b30e82e31660302a4af8cbe2f"`)
await db.query(`ALTER TABLE "erc1155_deposit" DROP CONSTRAINT "FK_732c4624c6d3a7de141edf6f732"`)
await db.query(`ALTER TABLE "input" DROP CONSTRAINT "FK_cf5fb31b5cf6719dc28adbb7d1b"`)
await db.query(`ALTER TABLE "application" DROP CONSTRAINT "FK_13e23630dd6edeb8dd7062483f5"`)
await db.query(`ALTER TABLE "application_factory" DROP CONSTRAINT "FK_402986e1692e825c3c773a56d62"`)
}
}
20 changes: 9 additions & 11 deletions docker-compose-db.yml
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
version: "3"

services:
db:
image: postgres:15
environment:
POSTGRES_DB: squid
POSTGRES_PASSWORD: postgres
ports:
- "${DB_PORT}:5432"
# command: ["postgres", "-c", "log_statement=all"]
shm_size: 1gb
db:
image: postgres:15
environment:
POSTGRES_DB: squid
POSTGRES_PASSWORD: postgres
ports:
- '${DB_PORT}:5432'
# command: ["postgres", "-c", "log_statement=all"]
shm_size: 1gb
Loading

0 comments on commit e215b72

Please sign in to comment.