feat: implement new apis and refactoring #28

zsystm · 2024-07-08T07:48:48Z

Implement new APIs

eth_getCode
eth_getStorageAt
eth_newFilter
eth_getFilterLogs
eth_newBlockFilter
eth_getFilterChanges
eth_uninstallFilter
eth_getLogs

Refactoring

Restricted the scope of access for each namespace API
- instead of accessing the backend directly, now access is through a scoped wrapper.
Renamed files and repositioned some methods within the backend directory to better reflect their purpose.

Fix

extractLogsFromEvents always returns an empty array
- evm log always emitted with EventTypeEVM
- there are no logs attributes for EventTypeCreate and EventTypeCall

Improvements for Future Consideration

In this PR, we followed the approach used in Ethermint, but it would be better to consider the following improvements in the future:
- Using an indexer to handle getFilterChanges instead of comet websocket connection and pubsub
- Preventing duplicate log filtering by storing filtered results

Bump-up cometBFT version

updated cometBFT version to initia-labs/cometbft v0.38.x which includes this commit
without the above commit, broadcasting tx always fails, cannot process any transactions.
- check the v0.38.x: spammy "discard late recheck response" for initial check cometbft/cometbft#3261 for details

License issue

This PR includes some codes that are derived from the Ethermint, which is licensed under the GNU Lesser General Public License (LGPL) v3. Minievm uses the Business Source License (BSL) 1.1, this could potentially lead to a license conflict.

Furth

Proof of works

setup minitiad for testing

unzip minitiad-setup.zip
mv minitiad-setup $HOME/.minitia

this setup includes rich account for testing
- bech32 addr: init1cml96vmptgw99syqrrz8az79xer2pcgptzssde
- eth addr: 0xc6fe5d33615a1c52c08018c47e8bc53646a0e101
- private key: 88cbead91aee890d27bf06e003ade3d4e952427e88f88d31d61d3ef5e5d54305
erc20 address of genesis coin umin: 0xb22eAAbf5a3377abDe553f23eA6d82709BA948E0

clone this branch and run make install
minitiad start

eth_getCode

Request: curl -X POST --data '{"jsonrpc":"2.0","method":"eth_getCode","params":["0xb22eAAbf5a3377abDe553f23eA6d82709BA948E0", "latest"],"id":1}' -H "Content-Type: application/json" http://localhost:8545
Response: {"jsonrpc":"2.0","id":1,"result":"0x60806...(skipped)"}

(skipped): I wrote ‘skipped’ for convenience, but the original return value is the bytecode of the contract. It was too long, so I shortened it.

eth_getStorageAt

Request:

curl -X POST -H "Content-Type: application/json" --data '{
  "jsonrpc":"2.0",
  "method":"eth_getStorageAt",
  "params": [
    "0xb22eAAbf5a3377abDe553f23eA6d82709BA948E0",
    "0x386c2fd1f87e34a6f8889cafe74cc4a0d53508a74e2ed92c2375fabcfd8563e9",
    "latest"
  ],
  "id":1
}' http://localhost:8545

Response: {"jsonrpc":"2.0","id":1,"result":"0x00000000000000000000000000000000000000000000021e19e0c9bab2400000"}

eth_newFilter

transfer umin to 0xc08d8192da03aafc8c516e35cc4b0cfd2c139f96 before calling this
above tx must included within scope (fromBlock ~ toBlock)

Request:

curl -X POST -H "Content-Type: application/json" --data '{
  "jsonrpc": "2.0",
  "method": "eth_newFilter",
  "params": [
    {
      "fromBlock": "0x2390",
      "toBlock": "0x2399",
      "address": "0xb22eAAbf5a3377abDe553f23eA6d82709BA948E0",
      "topics": [
        "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"
      ]
    }
  ],
  "id": 1
}' http://localhost:8545

Response: {"jsonrpc":"2.0","id":1,"result":"0xd6f8af542b4b3c699a354336358e4700"}

eth_getFilterLogs

Request: curl -X POST -H "Content-Type: application/json" --data '{"method":"eth_getFilterLogs","params":["0xd6f8af542b4b3c699a354336358e4700"],"id":1,"jsonrpc":"2.0"}' http://localhost:8545
Response:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": [
    {
      "address": "0xb22eAAbf5a3377abDe553f23eA6d82709BA948E0",
      "topics": [
        "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        "0x000000000000000000000000c6fe5d33615a1c52c08018c47e8bc53646a0e101",
        "0x00000000000000000000000025051c50bb6f302a7bd89100bead641e65da743b"
      ],
      "data": "0x0000000000000000000000000000000000000000000000000000000000000457",
      "blockNumber": "0x2398",
      "transactionHash": "0x80dcea3db9c181748bce112c760102b2f4d6ac7c25d514ec4ed16656b8fefdb6",
      "transactionIndex": "0x1",
      "blockHash": "0x6ebdf8f6a2ccf6c9b9aead31589524949e8e63c16e938f07100725e02c219432",
      "logIndex": "0x0",
      "removed": false
    }
  ]
}

eth_newBlockFilter

Request: curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_newBlockFilter","params":[],"id":1}' http://localhost:8545
Response: {"jsonrpc":"2.0","id":1,"result":"0x47bf8617648970968c2481aa433c2836"}

eth_getFilterChanges

wait some blocks to be created after create new block filter

Request: curl -X POST -H "Content-Type: application/json" --data '{"method":"eth_getFilterChanges","params":["0x47bf8617648970968c2481aa433c2836"],"id":1,"jsonrpc":"2.0"}' http://localhost:8545
Response: {"jsonrpc":"2.0","id":1,"result":["0x44c80c0041d93a5866a448a0d48762f8406e9df6354b099c2469137181d5649a"]}

eth_uninstallFilter

# Uninstall
curl -X POST  -H "Content-Type: application/json" --data '{"method":"eth_uninstallFilter","params":["0x47bf8617648970968c2481aa433c2836"],"id":1,"jsonrpc":"2.0"}' http://localhost:8545 
{"jsonrpc":"2.0","id":1,"result":true}

# Check the result of Uninstall
curl -X POST  -H "Content-Type: application/json" --data '{"method":"eth_uninstallFilter","params":["0x47bf8617648970968c2481aa433c2836"],"id":1,"jsonrpc":"2.0"}' http://localhost:8545 
{"jsonrpc":"2.0","id":1,"error":{"code":-32000,"message":"filter 0x47bf8617648970968c2481aa433c2836 not found"}}

eth_getLogs

Request: curl -X POST -H "Content-Type: application/json" --data '{"method":"eth_getLogs","params":[{"blockHash":"0x6ebdf8f6a2ccf6c9b9aead31589524949e8e63c16e938f07100725e02c219432"}],"id":1,"jsonrpc":"2.0"}' http://localhost:8545
Result:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": [
    {
      "address": "0xb22eAAbf5a3377abDe553f23eA6d82709BA948E0",
      "topics": [
        "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        "0x000000000000000000000000c6fe5d33615a1c52c08018c47e8bc53646a0e101",
        "0x00000000000000000000000025051c50bb6f302a7bd89100bead641e65da743b"
      ],
      "data": "0x0000000000000000000000000000000000000000000000000000000000000457",
      "blockNumber": "0x2398",
      "transactionHash": "0x80dcea3db9c181748bce112c760102b2f4d6ac7c25d514ec4ed16656b8fefdb6",
      "transactionIndex": "0x1",
      "blockHash": "0x6ebdf8f6a2ccf6c9b9aead31589524949e8e63c16e938f07100725e02c219432",
      "logIndex": "0x0",
      "removed": false
    }
  ]
}

new eth apis - GetStorageAt - GetCode refactor - Restricted the scope of access for each namespace API (instead of accessing the backend directly, now access is through a scoped wrapper) - Also renamed files and repositioned some methods within the backend directory to better reflect their purpose

temp disable txpool api because we don't have any implementation

Co-authored-by: zsystm <[email protected]>

ran the followings for this commit: - go mod tidy - go work sync

current version of cometbft which minievm uses have a bug(cometbft/cometbft#3264 (comment)) which makes transaction broadcasting unavailable.

dongsam

Aside from the comments regarding the cometBFT bump and the GNU license, LGTM.

dongsam · 2024-07-09T04:02:19Z

go.mod

@@ -280,7 +282,7 @@ replace (

 // initia custom
 replace (
-	github.com/cometbft/cometbft => github.com/initia-labs/cometbft v0.0.0-20240621094738-408dc5262680
+	github.com/cometbft/cometbft => github.com/initia-labs/cometbft v0.0.0-20240704071957-c46468756c01 //v0.0.0-20240621094738-408dc5262680


It seems that the target base branch and main branch of this PR are not yet using the specified version of cometBFT. Since there isn't a separate cometBFT bump PR, it would be helpful to add a more detailed explanation of the reason for the bump in this PR description. diff c46468756c01...408dc5262680

@dongsam
I have added the reasoning for bumping up cometBFT version.

updated cometBFT version to initia-labs/cometbft v0.38.x which includes this commit
without the above commit, broadcasting tx always fails, cannot process any transactions.
check the cometbft/cometbft#3261 for details

dongsam · 2024-07-09T04:08:02Z

jsonrpc/namespaces/eth/filters/filter_system.go

+// This file is part of Evmos' Ethermint library.
+//
+// The Ethermint library is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Lesser General Public License as published by


use code under the GNU license, there may be a requirement that the distribution must also follow the GNU license. However, since minievm currently uses a non-GNU license, it is important to clearly state in the PR description that GNU code from evmos/ethermint is being used. This will help the repository maintainer to be aware of this.

@dongsam
I have added License issue section to the PR description.

This PR includes some codes that are derived from the Ethermint, which is licensed under the GNU Lesser General Public License (LGPL) v3. Minievm uses the Business Source License (BSL) 1.1, this could potentially lead to a license conflict.

This can be problem hmm, I think we can rewrite filter logics to do not use comet but directly subscribe from indexer.

I will change this part after merging it.

beer-1

LGTM; left some comments but that parts will be changed in next PR, so let's merge it first and I will make new PR to change filter system to do not use comet.

beer-1 · 2024-07-10T06:57:40Z

jsonrpc/namespaces/eth/filters/api.go

+					continue
+				}
+
+				logs := evmtypes.Logs.ToEthLogs(ethResp.Logs)


I think Logs does not contain tx hash or block hash infos, so the subscriber has no way to retrieve the tx data or block data with this logs.

so maybe we need to fill the infos like GetLogsByHeight

for _, tx := range txs { receipt, err := b.app.EVMIndexer().TxReceiptByHash(queryCtx, tx.Hash) if err != nil { return nil, err } logs := receipt.Logs for idx, log := range logs { log.BlockHash = blockHeader.Hash() log.BlockNumber = height log.TxHash = tx.Hash log.Index = uint(idx) log.TxIndex = receipt.TransactionIndex } blockLogs = append(blockLogs, logs) }

also we don't know eth tx hash at this step, we need to convert cosmos tx to eth tx and get the eth tx hash and use that hash to get receipt.

beer-1 · 2024-07-10T06:59:40Z

jsonrpc/namespaces/eth/filters/api.go

+
+				api.filtersMu.Lock()
+				if f, found := api.filters[headerSub.ID()]; found {
+					f.hashes = append(f.hashes, common.BytesToHash(data.Header.Hash()))


cometbft block hash different from evm block hash, we try to mimic same logic with to generate block hash

minievm/indexer/abci.go

Line 101 in b6a073a

blockHash := blockHeader.Hash()

so good to get evm block hash with block number

beer-1 · 2024-07-10T07:50:23Z

jsonrpc/namespaces/eth/filters/filter_system.go

+// This file is part of Evmos' Ethermint library.
+//
+// The Ethermint library is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Lesser General Public License as published by


This can be problem hmm, I think we can rewrite filter logics to do not use comet but directly subscribe from indexer.

I will change this part after merging it.

* wip * wip2 * feat: json-rpc signing * direction * add indexer * enable jsonrpc * add precision checking logic * fix test * reterive value from the msg * fix: handle unit conversion in cosmos to ethereum tx (#24) whenever ethereum tx is converted into cosmos tx by ConvertEthereumTxToCosmosTx, the value is converted to cosmos fee unit from wei. so when convert the cosmos tx to back the original ethereum tx, we should convert the value back to wei to get original ethereum tx. * fix pubkey recover * fix test * feat: implement new apis and refactoring (#28) * feat: new eth apis, refactor: code structures new eth apis - GetStorageAt - GetCode refactor - Restricted the scope of access for each namespace API (instead of accessing the backend directly, now access is through a scoped wrapper) - Also renamed files and repositioned some methods within the backend directory to better reflect their purpose * fix: not implemented txpool api temp disable txpool api because we don't have any implementation * fix: change event type for extract logs * add cap for filters * add filter system and apis * fix comment Co-authored-by: zsystm <[email protected]> * delete unused interfaces and comments * fix varible and function naming * add TODO comments * delete unused functions * add error handling * Set websocket params to prevent timeout * fix: change GetLogsByHeight interface * chore: change variable names from tm to comet * docs: add comments about websocket client params * fix: go.mod and go.work.sum ran the followings for this commit: - go mod tidy - go work sync * update: cometbft dep current version of cometbft which minievm uses have a bug(cometbft/cometbft#3264 (comment)) which makes transaction broadcasting unavailable. --------- Co-authored-by: jason song <[email protected]> Co-authored-by: jasonsong0 <[email protected]> * feat: directly subscribe logs from indexer (#34) * change filter system to directly subscribe from indexer * use mutex and pass logs in bulk * fix dependency problems * add readme * update readme --------- Co-authored-by: suha jin <[email protected]> Co-authored-by: zsystm <[email protected]> Co-authored-by: jason song <[email protected]> Co-authored-by: jasonsong0 <[email protected]>

zsystm and others added 15 commits July 8, 2024 16:23

fix: not implemented txpool api

bc5e980

temp disable txpool api because we don't have any implementation

fix: change event type for extract logs

fc89773

add cap for filters

9e5e17c

add filter system and apis

f9412df

fix comment

c816b5c

Co-authored-by: zsystm <[email protected]>

delete unused interfaces and comments

af38813

fix varible and function naming

35b648d

add TODO comments

37b13b2

delete unused functions

c0a9e0e

add error handling

bc32eda

Set websocket params to prevent timeout

4e422f9

fix: change GetLogsByHeight interface

6cf230a

chore: change variable names from tm to comet

8d164f2

docs: add comments about websocket client params

f493a92

zsystm changed the title ~~feat: support essential json-rpc apis~~ feat: implement new apis and refactoring Jul 8, 2024

zsystm added 2 commits July 8, 2024 16:56

fix: go.mod and go.work.sum

88fc512

ran the followings for this commit: - go mod tidy - go work sync

update: cometbft dep

c463d44

current version of cometbft which minievm uses have a bug(cometbft/cometbft#3264 (comment)) which makes transaction broadcasting unavailable.

dongsam reviewed Jul 9, 2024

View reviewed changes

zsystm marked this pull request as ready for review July 9, 2024 06:16

beer-1 self-requested a review July 10, 2024 05:04

beer-1 approved these changes Jul 10, 2024

View reviewed changes

beer-1 merged commit 4d2f2a2 into initia-labs:feat/json-rpc Jul 10, 2024
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement new apis and refactoring #28

feat: implement new apis and refactoring #28

zsystm commented Jul 8, 2024 •

edited

Loading

dongsam left a comment

dongsam Jul 9, 2024

zsystm Jul 9, 2024

dongsam Jul 9, 2024

zsystm Jul 9, 2024

beer-1 Jul 10, 2024

beer-1 left a comment

beer-1 Jul 10, 2024

beer-1 Jul 10, 2024

beer-1 Jul 10, 2024

beer-1 Jul 10, 2024

beer-1 Jul 10, 2024

feat: implement new apis and refactoring #28

feat: implement new apis and refactoring #28

Conversation

zsystm commented Jul 8, 2024 • edited Loading

Implement new APIs

Refactoring

Fix

Improvements for Future Consideration

Bump-up cometBFT version

License issue

Furth

Proof of works

eth_getCode

eth_getStorageAt

eth_newFilter

eth_getFilterLogs

eth_newBlockFilter

eth_getFilterChanges

eth_uninstallFilter

eth_getLogs

dongsam left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beer-1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zsystm commented Jul 8, 2024 •

edited

Loading