From 87c2f4c2c93b0ffc57fc44505850bb629a434658 Mon Sep 17 00:00:00 2001 From: Roman Petriv Date: Wed, 3 Jan 2024 11:55:19 +0200 Subject: [PATCH] fix: tests for unit of work changes (#144) chore: fix dev discussions link (#143) - Updates dev discussions link - We updated the link - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. Co-authored-by: Vasyl Ivanchuk test: add allure public reports (#68) updated workflow to make public allure reports current reports provide a link to a private url - [ + ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ n/a ] Tests for the changes have been added / updated. - [ n/a ] Documentation comments have been added / updated. --------- Co-authored-by: Vasyl Ivanchuk feat: initial changes to show erc20 as native token in explorer feat: use nativechain instead of eth feat: finishing touches to show ERC20 as native token chore: rename .png chore: rename png asset feat: default to ethereum when rpc method is missing fix: for error of missing method fix: worker tests fix: worker tests fix: api tests and some e2e imports fix: move token.ts file to work for imports fix: common imports feat: adapt to base-token-addr-endpoint fix: worker tests fix: docs links (#150) Fix links to documentation. The documentation has been updated so we should use new links. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. feat: add data fetcher service (#145) Add data fetcher service that can be vertically scaled. Having data fetching vertically scaled we will be able to keep up with higher TPS. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. fix: remove portal from header (#159) - Remove Portal link from the top menu - Do not show an error reason for the failed transaction if it's empty - Docker compose starts services in production and not in development mode - Add zkVM filter option for solc compilers on the verification page To fix bug reports / implement feature requests. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. --------- Co-authored-by: Oleh Bairak <118197764+olehbairak@users.noreply.github.com> fix: add data fetcher debug logs (#165) Add more Data Fetcher debug logs. For easier troubleshooting. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. fix: fix data-fetcher tests chore: remove unused png chore: replace "Native Token" with "Base Token" chore: rename chainNative to baseToken feat: move base token fetching to separate module chore: remove old, unused code fix: use base token instead of base token fix: rename native token to base token chore: remove old, unused code fix: fix build, add comment fix: restore postgres default password on config feat: use base token from config in more places chore: update .env.example feat: move base-token fetching to a script fix: data-fetcher unit tests fix: api unit tests fix: worker tests fix: more tests chore: properly import config feat: rename to BaseToken feat: remove baseTokenData function feat: add base token address to configs feat: use base token from config instead of constant chore: remove unused scriped script fix: add transaction error reason transformer (#186) Add transaction error reason transformer. To unsure error and reason are transformed correctly. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). fix: add graceful shutdown timeout (#196) Add graceful shutdown timeout. To ensure that we can gracefully process all incoming HTTP requests before shutting down the service. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. feat: add transfer type filter for address transfers BFF API (#195) Add filter by transfer type to the address transfers endpoint It is useful to be able to get a list of transfers of certain type for address. For example get list of withdrawals. feat: add address transfer type migration (#199) - add script to migrate address transfer type - standard migration would not be able to complete, script uses parallel connections and updates data in small chunks. fix: use new DataSource instance for migration script (#200) fix: use logger in migration scripts (#201) fix: update address transfers type index order (#197) fix: update goerli deprecation message (#207) New goerli deprecation message with a specific deprecation date: image To notify users when the deprecation is gonna happen. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). fix: improve fetcher usage (#209) 1. Log timestamp in ISO8601 format. 2. Internal retries for `DataFetcherService`. 1. To have timestamps properly processed so logs are in the correct order. 2. Otherwise in case of an error the whole batch is retried which is unnecessary. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. fix: improve data fetcher response serialisation (#216) Get rid of the separate interceptor that transforms `BigNumber` to string, accomplish the same result overriding `BigNumber` `toJSON` function. Separate interceptor is not optimal especially for blocks with lots of related data, which we btw expect to see more often with higher TPS. There are blocks where all related info weights 2Mb+ and it takes ~200ms to transform the whole response object by the interceptor. Since after the transformation JSON serialisation happens anyway, we can just override `toJSON` for `BigNumber` and it will be picked up during JSON serialisation. I checked that with this change generated response is the same. Performance improvement for large payloads (2Mb+): 1. Serialisation with interceptor (interceptor + json serialiser): avg 290ms 2. Serialisation with overridden `toJSON` (json serialiser): avg 34ms But most important event loop is not blocked for a long time. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. feat: increase request body size limit (#217) feat: remove goerli network (#219) Remove goerli network. Clean up the codebase since we stop supporting Goerli network. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. - [X] Documentation comments have been added / updated. fix: update baseToken to baseTokenData in E2E tests fix(e2e tests): fix end-to-end test to use base token, modify eth address fix(e2e tests): fix eth address in address e2e tests fix(e2e tests): change missing apparences of l1 address fix(e2e tests): non existing var name fix: increase default timeout settings (#228) Increase default timeout settings. To make sure explorer can handle large transactions. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. fix(e2e tests): valid address format fix(e2e tests): add missing fields in select fix(e2e tests): remove unused modules fix(e2e tests): typecheck fix(e2e tests): increase amount of selects in test fix(e2e tests): add missing values in tests test(coverage): api config with base token test: uncomment tests fix: remove non existing script fix: address PR comments fix: lint errors fix: rpc method name fix: rollback address capitalization fix: use checksum address fix: rename root hash to block hash (#249) Rename root hash on the block screen to block hash. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. fix: use both version of the rpc response for getBridgeContract method Revert "fix(e2e tests): add missing fields in select" This reverts commit d3916ed83c87133e0b4fa8bde7e9e871bf66001d. Revert "fix(e2e tests): add missing fields in select" This reverts commit d3916ed83c87133e0b4fa8bde7e9e871bf66001d. fix(e2e): account tests fix: use correct amount of selects test: add tests for old response chore: fix docker compose (#256) Fix docker compose by applying the same updates that were done in [local-setup](https://github.com/matter-labs/local-setup.) Old docker compose doesn't work with the latest `local-node` docker image. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). fix: explorer doesn't require defined bridge address (#257) Fix explorer so that a defined bridge address is not required. In-memory node doesn't have L1 network and bridge addresses defined. This fix make explorer work with the in-memory node. - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. --- .dockerignore | 3 +- .../workflows/app-deploy-feature-branch.yml | 2 +- .github/workflows/app-deploy-preview.yml | 2 +- .github/workflows/app-e2e.yml | 73 +- .github/workflows/release.yml | 14 + .github/workflows/validate-pr.yml | 1 + .gitignore | 1 + .vscode/settings.json | 3 + README.md | 25 +- docker-compose-cli.yaml | 38 +- docker-compose.yaml | 77 +- package-lock.json | 211 +- packages/api/.env.example | 10 +- .../src/address/address.controller.spec.ts | 25 +- .../api/src/address/address.controller.ts | 17 +- .../dtos/filterAddressTransfersOptions.dto.ts | 13 + packages/api/src/address/dtos/index.ts | 1 + .../api/account/account.controller.spec.ts | 6 +- .../api/src/api/account/account.controller.ts | 6 +- .../mappers/internalTransactionMapper.spec.ts | 4 +- .../src/api/stats/stats.controller.spec.ts | 7 +- .../api/src/api/stats/stats.controller.ts | 7 +- .../src/api/token/token.controller.spec.ts | 25 +- .../api/src/api/token/token.controller.ts | 1 - packages/api/src/balance/balance.entity.ts | 13 +- packages/api/src/common/constants.ts | 2 +- packages/api/src/common/types.ts | 2 + .../config/docs/constants.testnet-goerli.json | 13 - packages/api/src/config/index.spec.ts | 69 +- packages/api/src/config/index.ts | 59 +- .../api/src/health/health.controller.spec.ts | 64 + packages/api/src/health/health.controller.ts | 32 +- packages/api/src/logger.ts | 8 +- packages/api/src/main.ts | 8 +- packages/api/src/token/token.entity.ts | 15 +- packages/api/src/token/token.module.ts | 1 - packages/api/src/token/token.service.spec.ts | 6 +- packages/api/src/token/token.service.ts | 12 +- .../src/transaction/transaction.controller.ts | 3 +- .../src/transfer/addressTransfer.entity.ts | 8 +- packages/api/src/transfer/transfer.entity.ts | 11 +- packages/api/src/transfer/transfer.module.ts | 1 - packages/api/src/transfer/transfer.service.ts | 3 +- packages/api/test/account-api.e2e-spec.ts | 8 +- packages/api/test/address.e2e-spec.ts | 37 +- packages/api/test/stats-api.e2e-spec.ts | 6 +- packages/api/test/token-api.e2e-spec.ts | 37 +- packages/api/test/token.e2e-spec.ts | 22 +- packages/api/test/transaction.e2e-spec.ts | 31 +- packages/app/README.md | 12 +- packages/app/mock/transactions/Execute.json | 10 +- .../app/mock/transactions/ExecuteFeeOnly.json | 8 +- packages/app/src/App.vue | 2 - packages/app/src/components/FeeData.vue | 9 +- .../app/src/components/NetworkDeprecated.vue | 44 - packages/app/src/components/TheFooter.vue | 2 +- .../app/src/components/balances/Table.vue | 8 +- .../app/src/components/blocks/InfoTable.vue | 6 +- .../common/CheckBoxInput.stories.ts | 29 + .../src/components/common/CheckBoxInput.vue | 42 + packages/app/src/components/form/FormItem.vue | 2 +- .../app/src/components/header/TheHeader.vue | 14 +- .../transactions/EthAmountPrice.vue | 5 +- .../app/src/components/transactions/Table.vue | 6 +- .../transactions/infoTable/GeneralInfo.vue | 2 +- .../infoTable/TransferTableCell.stories.ts | 2 +- packages/app/src/composables/useContext.ts | 10 +- .../src/composables/useEnvironmentConfig.ts | 7 + .../app/src/composables/useRuntimeConfig.ts | 21 +- packages/app/src/composables/useToken.ts | 3 - .../app/src/composables/useTokenLibrary.ts | 1 - packages/app/src/configs/dev.config.json | 37 +- packages/app/src/configs/index.ts | 4 +- packages/app/src/configs/local.config.json | 4 +- .../app/src/configs/production.config.json | 29 +- packages/app/src/configs/staging.config.json | 33 +- packages/app/src/locales/en.json | 10 +- packages/app/src/locales/uk.json | 5 +- packages/app/src/utils/constants.ts | 4 - .../src/views/ContractVerificationView.vue | 36 +- .../app/tests/components/AddressLink.spec.ts | 4 +- packages/app/tests/components/FeeData.spec.ts | 8 +- .../app/tests/components/NetworkStats.spec.ts | 2 +- .../tests/components/NetworkSwitch.spec.ts | 34 +- .../app/tests/components/TheFooter.spec.ts | 4 +- .../app/tests/components/TheHeader.spec.ts | 5 +- .../tests/components/balances/Table.spec.ts | 2 +- .../components/batches/InfoTable.spec.ts | 8 +- .../tests/components/blocks/InfoTable.spec.ts | 16 +- .../components/common/CheckBoxInput.spec.ts | 39 + .../transactions/GeneralInfo.spec.ts | 4 +- .../components/transactions/Status.spec.ts | 2 +- .../transactions/TransferInfo.spec.ts | 4 +- .../transactions/TransferTableCell.spec.ts | 2 +- .../tests/components/transfers/Table.spec.ts | 2 +- .../app/tests/composables/useContext.spec.ts | 24 +- .../tests/composables/useContractABI.spec.ts | 6 +- .../composables/useEnvironmentConfig.spec.ts | 10 +- .../tests/composables/useTokenLibrary.spec.ts | 8 +- .../tests/composables/useTransaction.spec.ts | 13 +- .../features/artifacts/artifactsSet1.feature | 51 +- .../features/artifacts/artifactsSet2.feature | 4 +- .../app/tests/e2e/features/copying.feature | 6 +- .../redirection/redirectionSet1.feature | 42 +- .../redirection/redirectionSet3.feature | 2 - packages/app/tests/e2e/src/data/data.ts | 1 - packages/app/tests/e2e/src/pages/base.page.ts | 2 +- packages/app/tests/mocks.ts | 36 +- .../views/ContractVerificationView.spec.ts | 50 +- packages/data-fetcher/.env.example | 15 + packages/data-fetcher/.eslintignore | 1 + packages/data-fetcher/.eslintrc.js | 25 + packages/data-fetcher/Dockerfile | 43 + packages/data-fetcher/README.md | 52 + packages/data-fetcher/nest-cli.json | 5 + packages/data-fetcher/package.json | 113 + packages/data-fetcher/src/abis/erc721.json | 333 +++ .../src/abis/l2StandardERC20.json | 64 + .../src/abis/transferEventWithNoIndexes.json | 0 .../src/address/address.service.spec.ts | 58 +- .../src/address/address.service.ts | 34 +- .../default.handler.spec.ts | 0 .../default.handler.ts | 0 .../extractContractDeployedHandlers/index.ts | 0 .../interface/contractAddress.interface.ts | 1 + ...extractContractAddressHandler.interface.ts | 0 packages/data-fetcher/src/app.module.ts | 38 + .../src/balance/balance.service.spec.ts | 534 ++++ .../src/balance/balance.service.ts | 122 + packages/data-fetcher/src/balance/index.ts | 1 + .../src/block/block.controller.spec.ts | 89 + .../src/block/block.controller.ts | 37 + .../src/block/block.service.spec.ts | 315 +++ .../data-fetcher/src/block/block.service.ts | 103 + packages/data-fetcher/src/block/index.ts | 2 + .../src/blockchain/blockchain.service.spec.ts | 2396 +++++++++++++++++ .../src/blockchain/blockchain.service.ts | 203 ++ packages/data-fetcher/src/blockchain/index.ts | 1 + .../src/blockchain/retryableContract.spec.ts | 267 ++ .../src/blockchain/retryableContract.ts | 124 + .../common/pipes/parseLimitedInt.pipe.spec.ts | 84 + .../src/common/pipes/parseLimitedInt.pipe.ts | 34 + packages/data-fetcher/src/config.spec.ts | 31 + packages/data-fetcher/src/config.ts | 37 + packages/data-fetcher/src/constants.ts | 38 + .../src/health/health.controller.spec.ts | 119 + .../src/health/health.controller.ts | 39 + .../data-fetcher/src/health/health.module.ts | 11 + .../src/health/jsonRpcProvider.health.spec.ts | 56 + .../src/health/jsonRpcProvider.health.ts | 22 + packages/data-fetcher/src/log/index.ts | 2 + .../data-fetcher/src/log/log.service.spec.ts | 150 ++ packages/data-fetcher/src/log/log.service.ts | 69 + packages/data-fetcher/src/log/logType.spec.ts | 22 + packages/data-fetcher/src/log/logType.ts | 54 + packages/data-fetcher/src/logger.ts | 29 + packages/data-fetcher/src/main.ts | 24 + packages/data-fetcher/src/metrics/index.ts | 2 + .../src/metrics/metrics.module.ts | 8 + .../src/metrics/metrics.provider.ts | 63 + .../data-fetcher/src/rpcProvider/index.ts | 4 + .../src/rpcProvider/jsonRpcProvider.module.ts | 57 + .../src/rpcProvider/jsonRpcProviderBase.ts | 7 + .../jsonRpcProviderExtended.spec.ts | 119 + .../rpcProvider/jsonRpcProviderExtended.ts | 64 + .../rpcProvider/webSocketProviderExtended.ts | 120 + .../rpcProvider/wrappedWebSocketProvider.ts | 56 + .../src/token/token.service.spec.ts | 523 ++++ .../data-fetcher/src/token/token.service.ts | 121 + .../data-fetcher/src/transaction/index.ts | 1 + .../transaction/transaction.service.spec.ts | 202 ++ .../src/transaction/transaction.service.ts | 90 + .../finalizeDeposit/default.handler.spec.ts | 11 +- .../finalizeDeposit/default.handler.ts | 12 +- .../src/transfer/extractHandlers/index.ts | 0 .../mint/ethMintFromL1.handler.spec.ts | 11 +- .../mint/ethMintFromL1.handler.ts | 15 +- .../contractDeployerTransfer.handler.spec.ts | 4 +- .../contractDeployerTransfer.handler.ts | 4 +- .../transfer/default.handler.spec.ts | 14 +- .../transfer/default.handler.ts | 7 +- .../transfer/erc721Transfer.handle.spec.ts | 4 +- .../transfer/erc721Transfer.handler.ts | 4 +- .../ethWithdrawalToL1.handler.spec.ts | 11 +- .../withdrawal/ethWithdrawalToL1.handler.ts | 14 +- .../default.handler.spec.ts | 11 +- .../withdrawalInitiated/default.handler.ts | 11 +- .../extractTransferHandler.interface.ts | 3 +- .../transfer/interfaces/transfer.interface.ts | 4 +- .../src/transfer/transfer.service.spec.ts | 512 ++-- .../src/transfer/transfer.service.ts | 30 +- packages/data-fetcher/src/utils/date.spec.ts | 8 + packages/data-fetcher/src/utils/date.ts | 1 + .../src/utils/isInternalTransaction.spec.ts | 14 +- .../src/utils/isInternalTransaction.ts | 7 +- .../src/utils/overrideBigNumberToJson.spec.ts | 12 + .../src/utils/overrideBigNumberToJson.ts | 9 + .../data-fetcher/src/utils/parseLog.spec.ts | 397 +++ packages/data-fetcher/src/utils/parseLog.ts | 63 + packages/data-fetcher/src/utils/token.ts | 4 + packages/data-fetcher/temp-response-new.json | 534 ++++ packages/data-fetcher/temp-response-old.json | 636 +++++ packages/data-fetcher/test/app.e2e-spec.ts | 25 + packages/data-fetcher/test/jest-e2e.json | 9 + .../test/logs/block-with-no-txs-logs.json | 18 + .../address-out-of-range.json | 80 + ...-deposit-from-l1-to-different-address.json | 103 + .../erc20/bridge-deposit-from-l1.json | 148 + .../transactionReceipts/erc20/deploy-l2.json | 107 + .../transactionReceipts/erc20/transfer.json | 62 + .../transactionReceipts/erc20/withdrawal.json | 136 + .../eth/deposit-no-fee.json | 83 + .../eth/deposit-to-different-address.json | 103 + .../eth/deposit-zero-value.json | 74 + .../test/transactionReceipts/eth/deposit.json | 103 + .../eth/transfer-to-zero-address.json | 62 + .../transactionReceipts/eth/transfer.json | 62 + .../eth/withdrawal-to-different-address.json | 125 + .../eth/withdrawal-zero-value.json | 106 + .../transactionReceipts/eth/withdrawal.json | 125 + .../erc20-transfer-to-zero-address.json | 47 + .../fee-with-no-deposits.json | 61 + .../greeter/deploy-to-l2.json | 63 + .../greeter/exec-set-greeting.json | 47 + .../log-parsing-error.json | 48 + .../eth-usdc-erc20-through-paymaster.json | 347 +++ .../eth-usdc-erc20-transfer.json | 287 ++ .../multiTransfer/multi-eth-transfer.json | 152 ++ .../sub-calls-to-other-contracts.json | 91 + .../test/transactionReceipts/nft/approve.json | 63 + .../transactionReceipts/nft/deploy-l2.json | 107 + .../test/transactionReceipts/nft/mint.json | 63 + .../transactionReceipts/nft/transfer.json | 63 + .../no-deposit-after-fee.json | 90 + .../no-matching-handlers.json | 91 + .../paymasters/transfer.json | 107 + .../pre-approved-erc20/deposit.json | 91 + .../pre-approved-erc20/transfer.json | 62 + .../withdrawal-to-diff-address.json | 121 + .../pre-approved-erc20/withdrawal.json | 121 + .../transactionReceipts/tx-with-no-logs.json | 45 + packages/data-fetcher/tsconfig.build.json | 4 + packages/data-fetcher/tsconfig.json | 22 + packages/worker/.env.example | 3 +- packages/worker/.eslintrc.js | 2 +- packages/worker/README.md | 3 +- .../migrationScripts/checkAddressBalances.js | 78 - .../migrationScripts/updateTransactions.js | 58 - packages/worker/package.json | 18 +- packages/worker/src/app.module.ts | 8 +- .../src/balance/balance.service.spec.ts | 624 +---- .../worker/src/balance/balance.service.ts | 124 +- .../worker/src/block/block.processor.spec.ts | 269 +- packages/worker/src/block/block.processor.ts | 89 +- packages/worker/src/block/block.service.ts | 1 + packages/worker/src/block/block.utils.spec.ts | 14 +- packages/worker/src/block/block.utils.ts | 4 +- .../worker/src/block/block.watcher.spec.ts | 14 +- packages/worker/src/block/block.watcher.ts | 25 +- .../src/blockchain/blockchain.service.spec.ts | 1647 +++++++++++ .../src/blockchain/blockchain.service.ts | 78 +- .../blocksRevert/blocksRevert.service.spec.ts | 8 +- packages/worker/src/config.spec.ts | 8 +- packages/worker/src/config.ts | 14 +- packages/worker/src/constants.ts | 27 - .../src/counter/counter.processor.spec.ts | 11 +- .../dataFetcher/dataFetcher.service.spec.ts | 124 + .../src/dataFetcher/dataFetcher.service.ts | 61 + packages/worker/src/dataFetcher/types.ts | 113 + .../src/entities/addressTransaction.entity.ts | 2 +- .../src/entities/addressTransfer.entity.ts | 15 +- .../worker/src/entities/balance.entity.ts | 6 +- packages/worker/src/entities/block.entity.ts | 18 +- packages/worker/src/entities/log.entity.ts | 2 +- packages/worker/src/entities/token.entity.ts | 2 +- .../worker/src/entities/transaction.entity.ts | 29 +- .../src/entities/transactionReceipt.entity.ts | 14 +- .../worker/src/entities/transfer.entity.ts | 15 +- packages/worker/src/log/index.ts | 2 - packages/worker/src/log/log.processor.spec.ts | 157 -- packages/worker/src/log/log.processor.ts | 66 - packages/worker/src/logger.ts | 8 +- .../worker/src/metrics/metrics.provider.ts | 6 - .../1709722093204-AddAddressTransferType.ts | 106 + .../1706115493392-BaseTokenEnumVariant.ts | 19 + .../1709722093204-AddAddressTransferType.ts | 23 + .../src/repositories/block.repository.spec.ts | 5 +- .../src/repositories/block.repository.ts | 3 +- .../rpcProvider/webSocketProviderExtended.ts | 2 +- .../worker/src/token/token.service.spec.ts | 58 +- packages/worker/src/token/token.service.ts | 2 +- .../transaction/transaction.processor.spec.ts | 268 +- .../src/transaction/transaction.processor.ts | 114 +- .../transformers/toLower.transformer.spec.ts | 29 - .../src/transformers/toLower.transformer.ts | 13 - .../transferFields.transformer.spec.ts | 46 - .../transferFields.transformer.ts | 22 - packages/worker/src/typeorm.config.ts | 1 - .../unitOfWork/unitOfWork.provider.spec.ts | 182 +- packages/worker/src/utils/date.spec.ts | 9 +- packages/worker/src/utils/date.ts | 2 + .../worker/src/utils/splitIntoChunks.spec.ts | 16 + reth_chaindata/reth_config | 86 + scripts/setup-hyperchain-config.ts | 15 +- 304 files changed, 17066 insertions(+), 2888 deletions(-) create mode 100644 .vscode/settings.json create mode 100644 packages/api/src/address/dtos/filterAddressTransfersOptions.dto.ts delete mode 100644 packages/api/src/config/docs/constants.testnet-goerli.json delete mode 100644 packages/app/src/components/NetworkDeprecated.vue create mode 100644 packages/app/src/components/common/CheckBoxInput.stories.ts create mode 100644 packages/app/src/components/common/CheckBoxInput.vue create mode 100644 packages/app/tests/components/common/CheckBoxInput.spec.ts create mode 100644 packages/data-fetcher/.env.example create mode 100644 packages/data-fetcher/.eslintignore create mode 100644 packages/data-fetcher/.eslintrc.js create mode 100644 packages/data-fetcher/Dockerfile create mode 100644 packages/data-fetcher/README.md create mode 100644 packages/data-fetcher/nest-cli.json create mode 100644 packages/data-fetcher/package.json create mode 100644 packages/data-fetcher/src/abis/erc721.json create mode 100644 packages/data-fetcher/src/abis/l2StandardERC20.json rename packages/{worker => data-fetcher}/src/abis/transferEventWithNoIndexes.json (100%) rename packages/{worker => data-fetcher}/src/address/address.service.spec.ts (60%) rename packages/{worker => data-fetcher}/src/address/address.service.ts (59%) rename packages/{worker => data-fetcher}/src/address/extractContractDeployedHandlers/default.handler.spec.ts (100%) rename packages/{worker => data-fetcher}/src/address/extractContractDeployedHandlers/default.handler.ts (100%) rename packages/{worker => data-fetcher}/src/address/extractContractDeployedHandlers/index.ts (100%) rename packages/{worker => data-fetcher}/src/address/interface/contractAddress.interface.ts (87%) rename packages/{worker => data-fetcher}/src/address/interface/extractContractAddressHandler.interface.ts (100%) create mode 100644 packages/data-fetcher/src/app.module.ts create mode 100644 packages/data-fetcher/src/balance/balance.service.spec.ts create mode 100644 packages/data-fetcher/src/balance/balance.service.ts create mode 100644 packages/data-fetcher/src/balance/index.ts create mode 100644 packages/data-fetcher/src/block/block.controller.spec.ts create mode 100644 packages/data-fetcher/src/block/block.controller.ts create mode 100644 packages/data-fetcher/src/block/block.service.spec.ts create mode 100644 packages/data-fetcher/src/block/block.service.ts create mode 100644 packages/data-fetcher/src/block/index.ts create mode 100644 packages/data-fetcher/src/blockchain/blockchain.service.spec.ts create mode 100644 packages/data-fetcher/src/blockchain/blockchain.service.ts create mode 100644 packages/data-fetcher/src/blockchain/index.ts create mode 100644 packages/data-fetcher/src/blockchain/retryableContract.spec.ts create mode 100644 packages/data-fetcher/src/blockchain/retryableContract.ts create mode 100644 packages/data-fetcher/src/common/pipes/parseLimitedInt.pipe.spec.ts create mode 100644 packages/data-fetcher/src/common/pipes/parseLimitedInt.pipe.ts create mode 100644 packages/data-fetcher/src/config.spec.ts create mode 100644 packages/data-fetcher/src/config.ts create mode 100644 packages/data-fetcher/src/constants.ts create mode 100644 packages/data-fetcher/src/health/health.controller.spec.ts create mode 100644 packages/data-fetcher/src/health/health.controller.ts create mode 100644 packages/data-fetcher/src/health/health.module.ts create mode 100644 packages/data-fetcher/src/health/jsonRpcProvider.health.spec.ts create mode 100644 packages/data-fetcher/src/health/jsonRpcProvider.health.ts create mode 100644 packages/data-fetcher/src/log/index.ts create mode 100644 packages/data-fetcher/src/log/log.service.spec.ts create mode 100644 packages/data-fetcher/src/log/log.service.ts create mode 100644 packages/data-fetcher/src/log/logType.spec.ts create mode 100644 packages/data-fetcher/src/log/logType.ts create mode 100644 packages/data-fetcher/src/logger.ts create mode 100644 packages/data-fetcher/src/main.ts create mode 100644 packages/data-fetcher/src/metrics/index.ts create mode 100644 packages/data-fetcher/src/metrics/metrics.module.ts create mode 100644 packages/data-fetcher/src/metrics/metrics.provider.ts create mode 100644 packages/data-fetcher/src/rpcProvider/index.ts create mode 100644 packages/data-fetcher/src/rpcProvider/jsonRpcProvider.module.ts create mode 100644 packages/data-fetcher/src/rpcProvider/jsonRpcProviderBase.ts create mode 100644 packages/data-fetcher/src/rpcProvider/jsonRpcProviderExtended.spec.ts create mode 100644 packages/data-fetcher/src/rpcProvider/jsonRpcProviderExtended.ts create mode 100644 packages/data-fetcher/src/rpcProvider/webSocketProviderExtended.ts create mode 100644 packages/data-fetcher/src/rpcProvider/wrappedWebSocketProvider.ts create mode 100644 packages/data-fetcher/src/token/token.service.spec.ts create mode 100644 packages/data-fetcher/src/token/token.service.ts create mode 100644 packages/data-fetcher/src/transaction/index.ts create mode 100644 packages/data-fetcher/src/transaction/transaction.service.spec.ts create mode 100644 packages/data-fetcher/src/transaction/transaction.service.ts rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/finalizeDeposit/default.handler.spec.ts (93%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/finalizeDeposit/default.handler.ts (72%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/index.ts (100%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/mint/ethMintFromL1.handler.spec.ts (93%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/mint/ethMintFromL1.handler.ts (75%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/transfer/contractDeployerTransfer.handler.spec.ts (98%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/transfer/contractDeployerTransfer.handler.ts (92%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/transfer/default.handler.spec.ts (95%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/transfer/default.handler.ts (87%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/transfer/erc721Transfer.handle.spec.ts (97%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/transfer/erc721Transfer.handler.ts (92%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/withdrawal/ethWithdrawalToL1.handler.spec.ts (93%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/withdrawal/ethWithdrawalToL1.handler.ts (75%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/withdrawalInitiated/default.handler.spec.ts (93%) rename packages/{worker => data-fetcher}/src/transfer/extractHandlers/withdrawalInitiated/default.handler.ts (72%) rename packages/{worker => data-fetcher}/src/transfer/interfaces/extractTransferHandler.interface.ts (77%) rename packages/{worker => data-fetcher}/src/transfer/interfaces/transfer.interface.ts (78%) rename packages/{worker => data-fetcher}/src/transfer/transfer.service.spec.ts (82%) rename packages/{worker => data-fetcher}/src/transfer/transfer.service.ts (87%) create mode 100644 packages/data-fetcher/src/utils/date.spec.ts create mode 100644 packages/data-fetcher/src/utils/date.ts rename packages/{worker => data-fetcher}/src/utils/isInternalTransaction.spec.ts (87%) rename packages/{worker => data-fetcher}/src/utils/isInternalTransaction.ts (67%) create mode 100644 packages/data-fetcher/src/utils/overrideBigNumberToJson.spec.ts create mode 100644 packages/data-fetcher/src/utils/overrideBigNumberToJson.ts create mode 100644 packages/data-fetcher/src/utils/parseLog.spec.ts create mode 100644 packages/data-fetcher/src/utils/parseLog.ts create mode 100644 packages/data-fetcher/src/utils/token.ts create mode 100644 packages/data-fetcher/temp-response-new.json create mode 100644 packages/data-fetcher/temp-response-old.json create mode 100644 packages/data-fetcher/test/app.e2e-spec.ts create mode 100644 packages/data-fetcher/test/jest-e2e.json create mode 100644 packages/data-fetcher/test/logs/block-with-no-txs-logs.json create mode 100644 packages/data-fetcher/test/transactionReceipts/address-out-of-range.json create mode 100644 packages/data-fetcher/test/transactionReceipts/erc20/bridge-deposit-from-l1-to-different-address.json create mode 100644 packages/data-fetcher/test/transactionReceipts/erc20/bridge-deposit-from-l1.json create mode 100644 packages/data-fetcher/test/transactionReceipts/erc20/deploy-l2.json create mode 100644 packages/data-fetcher/test/transactionReceipts/erc20/transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/erc20/withdrawal.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/deposit-no-fee.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/deposit-to-different-address.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/deposit-zero-value.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/deposit.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/transfer-to-zero-address.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/withdrawal-to-different-address.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/withdrawal-zero-value.json create mode 100644 packages/data-fetcher/test/transactionReceipts/eth/withdrawal.json create mode 100644 packages/data-fetcher/test/transactionReceipts/failedTx/erc20-transfer-to-zero-address.json create mode 100644 packages/data-fetcher/test/transactionReceipts/fee-with-no-deposits.json create mode 100644 packages/data-fetcher/test/transactionReceipts/greeter/deploy-to-l2.json create mode 100644 packages/data-fetcher/test/transactionReceipts/greeter/exec-set-greeting.json create mode 100644 packages/data-fetcher/test/transactionReceipts/log-parsing-error.json create mode 100644 packages/data-fetcher/test/transactionReceipts/multiTransfer/eth-usdc-erc20-through-paymaster.json create mode 100644 packages/data-fetcher/test/transactionReceipts/multiTransfer/eth-usdc-erc20-transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/multiTransfer/multi-eth-transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/nestedContractsCalls/sub-calls-to-other-contracts.json create mode 100644 packages/data-fetcher/test/transactionReceipts/nft/approve.json create mode 100644 packages/data-fetcher/test/transactionReceipts/nft/deploy-l2.json create mode 100644 packages/data-fetcher/test/transactionReceipts/nft/mint.json create mode 100644 packages/data-fetcher/test/transactionReceipts/nft/transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/no-deposit-after-fee.json create mode 100644 packages/data-fetcher/test/transactionReceipts/no-matching-handlers.json create mode 100644 packages/data-fetcher/test/transactionReceipts/paymasters/transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/pre-approved-erc20/deposit.json create mode 100644 packages/data-fetcher/test/transactionReceipts/pre-approved-erc20/transfer.json create mode 100644 packages/data-fetcher/test/transactionReceipts/pre-approved-erc20/withdrawal-to-diff-address.json create mode 100644 packages/data-fetcher/test/transactionReceipts/pre-approved-erc20/withdrawal.json create mode 100644 packages/data-fetcher/test/transactionReceipts/tx-with-no-logs.json create mode 100644 packages/data-fetcher/tsconfig.build.json create mode 100644 packages/data-fetcher/tsconfig.json delete mode 100644 packages/worker/migrationScripts/checkAddressBalances.js delete mode 100644 packages/worker/migrationScripts/updateTransactions.js create mode 100644 packages/worker/src/blockchain/blockchain.service.spec.ts create mode 100644 packages/worker/src/dataFetcher/dataFetcher.service.spec.ts create mode 100644 packages/worker/src/dataFetcher/dataFetcher.service.ts create mode 100644 packages/worker/src/dataFetcher/types.ts delete mode 100644 packages/worker/src/log/index.ts delete mode 100644 packages/worker/src/log/log.processor.spec.ts delete mode 100644 packages/worker/src/log/log.processor.ts create mode 100644 packages/worker/src/migrationScripts/1709722093204-AddAddressTransferType.ts create mode 100644 packages/worker/src/migrations/1706115493392-BaseTokenEnumVariant.ts create mode 100644 packages/worker/src/migrations/1709722093204-AddAddressTransferType.ts delete mode 100644 packages/worker/src/transformers/toLower.transformer.spec.ts delete mode 100644 packages/worker/src/transformers/toLower.transformer.ts delete mode 100644 packages/worker/src/transformers/transferFields.transformer.spec.ts delete mode 100644 packages/worker/src/transformers/transferFields.transformer.ts create mode 100644 packages/worker/src/utils/splitIntoChunks.spec.ts create mode 100644 reth_chaindata/reth_config diff --git a/.dockerignore b/.dockerignore index 20b25939ec..f05f8a34ae 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1 +1,2 @@ -**/dist/ \ No newline at end of file +**/dist/ +**.env \ No newline at end of file diff --git a/.github/workflows/app-deploy-feature-branch.yml b/.github/workflows/app-deploy-feature-branch.yml index 8d3debee5a..e7bc4fca09 100644 --- a/.github/workflows/app-deploy-feature-branch.yml +++ b/.github/workflows/app-deploy-feature-branch.yml @@ -70,7 +70,7 @@ jobs: uses: ./.github/workflows/app-e2e.yml secrets: inherit permissions: - contents: read + contents: write with: targetUrl: ${{ needs.build.outputs.dappUrl }} testnet_network_value_for_e2e: "/?network=sepolia" diff --git a/.github/workflows/app-deploy-preview.yml b/.github/workflows/app-deploy-preview.yml index dc14276de3..b2208e0662 100644 --- a/.github/workflows/app-deploy-preview.yml +++ b/.github/workflows/app-deploy-preview.yml @@ -67,7 +67,7 @@ jobs: uses: ./.github/workflows/app-e2e.yml secrets: inherit permissions: - contents: read + contents: write with: targetUrl: ${{ needs.deploy.outputs.dappUrl }} testnet_network_value_for_e2e: "/?network=sepolia" diff --git a/.github/workflows/app-e2e.yml b/.github/workflows/app-e2e.yml index 354c768469..0e56d22699 100644 --- a/.github/workflows/app-e2e.yml +++ b/.github/workflows/app-e2e.yml @@ -68,6 +68,12 @@ jobs: with: fetch-depth: 0 + - name: Setup Node.js + uses: actions/setup-node@v3 + with: + node-version: '18' + cache: 'npm' + - name: Cache node modules id: cache-nodemodules uses: actions/cache@v3 @@ -110,27 +116,21 @@ jobs: E2ENETWORK='${{ inputs.default_network_value_for_e2e }}' npx cucumber-js --tags "${{ matrix.tags }} ${{ inputs.environmentTags }} and not @testnet" fi - - name: Reset tags quotes + - name: Save artifacts to Git if: always() - run: | - echo "MATRIX_TAG_WITHOUT_QUOTES=$(echo ${{ matrix.tags }} | sed -e 's/@//g' )" >> $GITHUB_ENV + uses: actions/upload-artifact@v3 + with: + name: allure-results + path: packages/app/allure-results - - name: Create launch ID + - name: Upload test results to Allure reporter if: always() - env: - ALLURE_LAUNCH_NAME: "#${{ github.run_number }} ${{ env.MATRIX_TAG_WITHOUT_QUOTES }}" - ALLURE_LAUNCH_TAGS: "${{ env.ALLURE_BASIC_TAGS }}, ${{ env.MATRIX_TAG_WITHOUT_QUOTES }}" - ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }} - run: | - echo "ALLURE_LAUNCH_ID=$(./allurectl launch create --launch-name '${{ env.ALLURE_LAUNCH_NAME }}' --no-header --format ID | tail -n1)" >> $GITHUB_ENV - - - name: Upload tests to the Allure proj - if: always() && inputs.publish_to_allure == true env: ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }} run: | - ./allurectl upload allure-results --launch-id ${{ env.ALLURE_LAUNCH_ID }} - ./allurectl launch close ${{ env.ALLURE_LAUNCH_ID }} + ./allurectl upload allure-results + echo "*Public report link: https://raw.githack.com/matter-labs/block-explorer/gh-pages/${{ github.run_number }}/index.html" + echo "*Public report link will be available when the 'Allure Report' job will be succesfully executed." - if: failure() name: Save artifacts @@ -140,18 +140,53 @@ jobs: path: packages/app/tests/e2e/artifacts/* publish: - name: Publish Allure link to GIT + name: Allure Report runs-on: ubuntu-latest permissions: - contents: read + contents: write needs: e2e if: always() steps: + - uses: actions/checkout@v3 + + - uses: actions/download-artifact@v2 + with: + name: allure-results + path: packages/app/allure-results + + - name: Get Allure history + uses: actions/checkout@v3 + if: always() + continue-on-error: true + with: + ref: gh-pages + path: gh-pages + + - name: Allure Report action from marketplace + uses: simple-elf/allure-report-action@v1.7 + if: always() + id: allure-report + with: + allure_results: packages/app/allure-results + gh_pages: gh-pages + allure_report: allure-report + allure_history: allure-history + keep_reports: 10 + + - name: Deploy report to Github Pages + if: always() + uses: peaceiris/actions-gh-pages@v2 + env: + PERSONAL_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PUBLISH_BRANCH: gh-pages + PUBLISH_DIR: allure-history + - name: Prepare a link run: | echo "BASE64_SEARCH_REQUEST=$(echo '${{ env.ALLURE_SEARCH_REQUEST }}' | base64)" >> $GITHUB_ENV - name: Publish Allure link to GIT Summary run: | - LINK="${{ vars.ALLURE_ENDPOINT }}project/${{ vars.ALLURE_PROJECT_ID }}/launches?search=${{ env.BASE64_SEARCH_REQUEST }}" - echo "Allure [e2e tests]($LINK) :rocket: in git run #${{ github.run_number }}" >> $GITHUB_STEP_SUMMARY + LINK1="${{ vars.ALLURE_ENDPOINT }}project/${{ vars.ALLURE_PROJECT_ID }}/launches?search=${{ env.BASE64_SEARCH_REQUEST }}" + LINK2="https://raw.githack.com/matter-labs/block-explorer/gh-pages/${{ github.run_number }}/index.html" + echo "Allure [Private]($LINK1) and [Public]($LINK2) links:rocket: in git run #${{ github.run_number }}" >> $GITHUB_STEP_SUMMARY diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 74b9091810..8415c96558 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -111,6 +111,20 @@ jobs: file: packages/worker/Dockerfile no-cache: true + - name: Build and push Docker image for Data Fetcher + uses: docker/build-push-action@v4 + with: + push: true + tags: | + "matterlabs/block-explorer-data-fetcher:latest" + "matterlabs/block-explorer-data-fetcher:v${{ needs.createReleaseVersion.outputs.releaseVersion }}" + "matterlabs/block-explorer-data-fetcher:${{ steps.setVersionForFlux.outputs.imageTag }}" + "us-docker.pkg.dev/matterlabs-infra/matterlabs-docker/block-explorer-data-fetcher:latest" + "us-docker.pkg.dev/matterlabs-infra/matterlabs-docker/block-explorer-data-fetcher:v${{ needs.createReleaseVersion.outputs.releaseVersion }}" + "us-docker.pkg.dev/matterlabs-infra/matterlabs-docker/block-explorer-data-fetcher:${{ steps.setVersionForFlux.outputs.imageTag }}" + file: packages/data-fetcher/Dockerfile + no-cache: true + - name: Build and push Docker image for App uses: docker/build-push-action@v4 with: diff --git a/.github/workflows/validate-pr.yml b/.github/workflows/validate-pr.yml index 539a8e0679..a837a4a49e 100644 --- a/.github/workflows/validate-pr.yml +++ b/.github/workflows/validate-pr.yml @@ -66,6 +66,7 @@ jobs: packages/app/junit.xml packages/api/junit.xml packages/worker/junit.xml + packages/data-fetcher/junit.xml check_run_annotations: all tests, skipped tests report_individual_runs: "true" check_name: Unit Test Results diff --git a/.gitignore b/.gitignore index ecbf0116b8..50a0a84644 100644 --- a/.gitignore +++ b/.gitignore @@ -22,6 +22,7 @@ tests/e2e/reports/ # Logs logs !/packages/worker/test/logs/ +!/packages/data-fetcher/test/logs/ *.log npm-debug.log* yarn-debug.log* diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 0000000000..55712c19f1 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,3 @@ +{ + "typescript.tsdk": "node_modules/typescript/lib" +} \ No newline at end of file diff --git a/README.md b/README.md index e9276bde56..43eabb9129 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,9 @@

Online blockchain browser for viewing and analyzing zkSync Era blockchain.

## 📌 Overview -This repository is a monorepo consisting of 3 packages: -- [Worker](./packages/worker) - an indexer service for [zkSync Era](https://zksync.io) blockchain data. The purpose of the service is to read the data from the blockchain in real time, transform it and fill in it's database with the data in a way that makes it easy to be queried by the [API](./packages/api) service. +This repository is a monorepo consisting of 4 packages: +- [Worker](./packages/worker) - an indexer service for [zkSync Era](https://zksync.io) blockchain data. The purpose of the service is to read blockchain data in real time, transform it and fill in it's database with the data in a way that makes it easy to be queried by the [API](./packages/api) service. +- [Data Fetcher](./packages/data-fetcher) - a service that exposes and implements an HTTP endpoint to retrieve aggregated data for a certain block / range of blocks from the blockchain. This endpoint is called by the [Worker](./packages/worker) service. - [API](./packages/api) - a service providing Web API for retrieving structured [zkSync Era](https://zksync.io) blockchain data collected by [Worker](./packages/worker). It connects to the Worker's database to be able to query the collected data. - [App](./packages/app) - a front-end app providing an easy-to-use interface for users to view and inspect transactions, blocks, contracts and more. It makes requests to the [API](./packages/api) to get the data and presents it in a way that's easy to read and understand. @@ -20,10 +21,14 @@ flowchart subgraph explorer[Block explorer] Database[("Block explorer DB
(PostgreSQL)")] Worker(Worker service) + Data-Fetcher(Data Fetcher service) API(API service) App(App) - + + Worker-."Request aggregated data (HTTP)".->Data-Fetcher + Data-Fetcher-."Request data (HTTP)".->Blockchain Worker-.Save processed data.->Database + API-.Query data.->Database App-."Request data (HTTP)".->API App-."Request data (HTTP)".->Blockchain @@ -32,7 +37,7 @@ flowchart Worker-."Request data (HTTP)".->Blockchain ``` -[Worker](./packages/worker) service is responsible for getting data from blockchain using [zkSync Era JSON-RPC API](https://era.zksync.io/docs/api/api.html), processing it and saving into the database. [API](./packages/api) service is connected to the same database where it gets the data from to handle API requests. It performs only read requests to the database. The front-end [App](./packages/app) makes HTTP calls to the Block Explorer [API](./packages/api) to get blockchain data and to the [zkSync Era JSON-RPC API](https://era.zksync.io/docs/api/api.html) for reading contracts, performing transactions etc. +[Worker](./packages/worker) service retrieves aggregated data from the [Data Fetcher](./packages/data-fetcher) via HTTP and also directly from the blockchain using [zkSync Era JSON-RPC API](https://era.zksync.io/docs/api/api.html), processes it and saves into the database. [API](./packages/api) service is connected to the same database where it gets the data from to handle API requests. It performs only read requests to the database. The front-end [App](./packages/app) makes HTTP calls to the Block Explorer [API](./packages/api) to get blockchain data and to the [zkSync Era JSON-RPC API](https://era.zksync.io/docs/api/api.html) for reading contracts, performing transactions etc. ## 🚀 Features @@ -56,12 +61,12 @@ npm install ## ⚙️ Setting up env variables ### Manually set up env variables -Make sure you have set up all the necessary env variables. Follow [Setting up env variables for Worker](./packages/worker#setting-up-env-variables) and [Setting up env variables for API](./packages/api#setting-up-env-variables) for instructions. For the [App](./packages/app) package you might want to edit environment config, see [Environment configs](./packages/app#environment-configs). +Make sure you have set up all the necessary env variables. Follow setting up env variables instructions for [Worker](./packages/worker#setting-up-env-variables), [Data Fetcher](./packages/data-fetcher#setting-up-env-variables) and [API](./packages/api#setting-up-env-variables). For the [App](./packages/app) package you might want to edit environment config, see [Environment configs](./packages/app#environment-configs). ### Build env variables based on your [zksync-era](https://github.com/matter-labs/zksync-era) local repo setup Make sure you have [zksync-era](https://github.com/matter-labs/zksync-era) repo set up locally. You must have your environment variables files present in the [zksync-era](https://github.com/matter-labs/zksync-era) repo at `/etc/env/*.env` for the build envs script to work. -The following script sets `.env` files for [Worker](./packages/worker) and [API](./packages/api) packages as well as environment configuration file for [App](./packages/app) package based on your local [zksync-era](https://github.com/matter-labs/zksync-era) repo setup. +The following script sets `.env` files for [Worker](./packages/worker), [Data Fetcher](./packages/data-fetcher) and [API](./packages/api) packages as well as environment configuration file for [App](./packages/app) package based on your local [zksync-era](https://github.com/matter-labs/zksync-era) repo setup. ```bash npm run hyperchain:configure ``` @@ -75,7 +80,7 @@ To create a database run the following command: npm run db:create ``` -To run all the packages (`Worker`, `API` and front-end `App`) in `development` mode run the following command from the root directory. +To run all the packages (`Worker`, `Data Fetcher`, `API` and front-end `App`) in `development` mode run the following command from the root directory. ```bash npm run dev ``` @@ -91,7 +96,7 @@ Each component can also be started individually. Follow individual packages `REA ## 🐳 Running in Docker There is a docker compose configuration that allows you to run Block Explorer and all its dependencies in docker. Just run the following command to spin up the whole environment: ``` -docker-compose up +docker compose up ``` It will run local Ethereum node, ZkSync Era, Postgres DB and all Block Explorer services. @@ -100,7 +105,7 @@ To get block-explorer connected to your ZK Stack Hyperchain you need to set up a ## 🔍 Verify Block Explorer is up and running -To verify front-end `App` is running open http://localhost:3010 in your browser. `API` should be available at http://localhost:3020. `Worker` - http://localhost:3001. +To verify front-end `App` is running open http://localhost:3010 in your browser. `API` should be available at http://localhost:3020, `Worker` at http://localhost:3001 and `Data Fetcher` at http://localhost:3040. ## 🕵️‍♂️ Testing Run unit tests for all packages: @@ -129,9 +134,7 @@ zkSync Era Block Explorer is distributed under the terms of either at your option. ## 🔗 Production links -- Testnet Goerli API: https://block-explorer-api.testnets.zksync.dev - Testnet Sepolia API: https://block-explorer-api.sepolia.zksync.dev - Mainnet API: https://block-explorer-api.mainnet.zksync.io -- Testnet Goerli App: https://goerli.explorer.zksync.io - Testnet Sepolia App: https://sepolia.explorer.zksync.io - Mainnet App: https://explorer.zksync.io diff --git a/docker-compose-cli.yaml b/docker-compose-cli.yaml index 1343eb8fa2..a8f29e8478 100644 --- a/docker-compose-cli.yaml +++ b/docker-compose-cli.yaml @@ -1,10 +1,7 @@ -version: '3.2' - services: app: - build: - context: . - dockerfile: ./packages/app/Dockerfile + platform: linux/amd64 + image: "matterlabs/block-explorer-app:${VERSION}" ports: - '3010:3010' depends_on: @@ -12,9 +9,8 @@ services: restart: unless-stopped worker: - build: - context: . - dockerfile: ./packages/worker/Dockerfile + platform: linux/amd64 + image: "matterlabs/block-explorer-worker:${VERSION}" environment: - PORT=3001 - LOG_LEVEL=verbose @@ -23,14 +19,30 @@ services: - DATABASE_USER=postgres - DATABASE_PASSWORD=postgres - DATABASE_NAME=block-explorer - - BLOCKCHAIN_RPC_URL=http://host.docker.internal:3050 + - BLOCKCHAIN_RPC_URL=http://host.docker.internal:${RPC_PORT} + - DATA_FETCHER_URL=http://data-fetcher:3040 - BATCHES_PROCESSING_POLLING_INTERVAL=1000 restart: unless-stopped + extra_hosts: + - "host.docker.internal:host-gateway" + + data-fetcher: + platform: linux/amd64 + image: "matterlabs/block-explorer-data-fetcher:${VERSION}" + environment: + - PORT=3040 + - LOG_LEVEL=verbose + - NODE_ENV=development + - BLOCKCHAIN_RPC_URL=http://host.docker.internal:${RPC_PORT} + ports: + - '3040:3040' + restart: unless-stopped + extra_hosts: + - "host.docker.internal:host-gateway" api: - build: - context: . - dockerfile: ./packages/api/Dockerfile + platform: linux/amd64 + image: "matterlabs/block-explorer-api:${VERSION}" environment: - PORT=3020 - METRICS_PORT=3005 @@ -60,4 +72,4 @@ services: - POSTGRES_DB=block-explorer volumes: - postgres: + postgres: \ No newline at end of file diff --git a/docker-compose.yaml b/docker-compose.yaml index cd747b2e49..390d5a3bad 100644 --- a/docker-compose.yaml +++ b/docker-compose.yaml @@ -1,16 +1,10 @@ -name: block-explorer services: app: build: context: . dockerfile: ./packages/app/Dockerfile - target: development-stage - command: npm run --prefix packages/app dev -- --host ports: - '3010:3010' - volumes: - - ./packages/app:/usr/src/app/packages/app - - /usr/src/app/packages/app/node_modules depends_on: - api restart: unless-stopped @@ -19,25 +13,33 @@ services: build: context: . dockerfile: ./packages/worker/Dockerfile - target: development-stage - command: npm run --prefix packages/worker dev:debug environment: - PORT=3001 - LOG_LEVEL=verbose - - NODE_ENV=development - DATABASE_HOST=postgres - DATABASE_USER=postgres - DATABASE_PASSWORD=postgres - DATABASE_NAME=block-explorer - BLOCKCHAIN_RPC_URL=http://zksync:3050 + - DATA_FETCHER_URL=http://data-fetcher:3040 - BATCHES_PROCESSING_POLLING_INTERVAL=1000 ports: - '3001:3001' - - '9229:9229' - - '9230:9230' - volumes: - - ./packages/worker:/usr/src/app/packages/worker - - /usr/src/app/packages/worker/node_modules + depends_on: + zksync: + condition: service_healthy + restart: unless-stopped + + data-fetcher: + build: + context: . + dockerfile: ./packages/data-fetcher/Dockerfile + environment: + - PORT=3040 + - LOG_LEVEL=verbose + - BLOCKCHAIN_RPC_URL=http://zksync:3050 + ports: + - '3040:3040' depends_on: zksync: condition: service_healthy @@ -47,22 +49,14 @@ services: build: context: . dockerfile: ./packages/api/Dockerfile - target: development-stage - command: npm run --prefix packages/api dev:debug environment: - PORT=3020 - METRICS_PORT=3005 - LOG_LEVEL=verbose - - NODE_ENV=development - DATABASE_URL=postgres://postgres:postgres@postgres:5432/block-explorer ports: - '3020:3020' - '3005:3005' - - '9231:9229' - - '9232:9230' - volumes: - - ./packages/api:/usr/src/app/packages/api - - /usr/src/app/packages/api/node_modules depends_on: - worker restart: unless-stopped @@ -85,15 +79,18 @@ services: - POSTGRES_PASSWORD=postgres - POSTGRES_DB=block-explorer - geth: - image: "matterlabs/geth:latest" - logging: - driver: none - ports: - - "8545:8545" - - "8546:8546" + reth: + restart: always + image: "ghcr.io/paradigmxyz/reth:v0.2.0-beta.2" volumes: - - geth:/var/lib/geth/data + - type: bind + source: ./reth_chaindata + target: /chaindata + command: node --dev --datadir /rethdata --http --http.addr 0.0.0.0 --http.port 8545 --dev.block-time 300ms --chain /chaindata/reth_config + environment: + - RUST_LOG=warn + ports: + - 127.0.0.1:8545:8545 zksync: stdin_open: true @@ -102,28 +99,30 @@ services: depends_on: postgres: condition: service_healthy - geth: + reth: condition: service_started ports: - - "3050:3050" # JSON RPC HTTP port - - "3051:3051" # JSON RPC WS port + - 127.0.0.1:3050:3050 # JSON RPC HTTP port + - 127.0.0.1:3051:3051 # JSON RPC WS port volumes: # Configs folder bind - zksync-config:/etc/env/ # Storage folder bind - zksync-data:/var/lib/zksync/data environment: - - DATABASE_URL=postgres://postgres:postgres@postgres:5432/zksync_local - - ETH_CLIENT_WEB3_URL=http://geth:8545 + - DATABASE_PROVER_URL=postgres://postgres:postgres@postgres/prover_local + - DATABASE_URL=postgres://postgres:postgres@postgres/zksync_local + - ETH_CLIENT_WEB3_URL=http://reth:8545 healthcheck: - test: "curl -H \"Content-Type: application/json\" -X POST --data '{\"jsonrpc\":\"2.0\",\"method\":\"web3_clientVersion\",\"params\":[],\"id\":67}' 127.0.0.1:3050 || exit 1" - interval: 5s + test: curl --fail http://localhost:3071/health || exit 1 + interval: 10s timeout: 5s - retries: 120 + retries: 60 + start_period: 30s restart: unless-stopped volumes: - geth: + reth: postgres: zksync-config: zksync-data: \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index b5e9876c27..48f4a732c9 100644 --- a/package-lock.json +++ b/package-lock.json @@ -17614,9 +17614,9 @@ } }, "node_modules/@types/yargs": { - "version": "17.0.29", - "resolved": "https://registry.npmjs.org/@types/yargs/-/yargs-17.0.29.tgz", - "integrity": "sha512-nacjqA3ee9zRF/++a3FUY1suHTFKZeHba2n8WeDw9cCVdmzmHpIxyzOJBcpHvvEmS8E9KqWlSnWHUkOrkhWcvA==", + "version": "17.0.32", + "resolved": "https://registry.npmjs.org/@types/yargs/-/yargs-17.0.32.tgz", + "integrity": "sha512-xQ67Yc/laOG5uMfX/093MRlGGCIBzZMarVa+gfNKJxWAIgykYpVGkBdbqEzGDDfCrVUj6Hiff4mTZ5BA6TmAog==", "dependencies": { "@types/yargs-parser": "*" } @@ -23466,6 +23466,10 @@ "node": ">=8" } }, + "node_modules/data-fetcher": { + "resolved": "packages/data-fetcher", + "link": true + }, "node_modules/data-urls": { "version": "3.0.2", "resolved": "https://registry.npmjs.org/data-urls/-/data-urls-3.0.2.tgz", @@ -55159,6 +55163,205 @@ "typedarray-to-buffer": "^3.1.5" } }, + "packages/data-fetcher": { + "version": "0.0.0", + "license": "MIT", + "dependencies": { + "@nestjs/common": "^9.0.0", + "@nestjs/config": "^2.2.0", + "@nestjs/core": "^9.0.0", + "@nestjs/platform-express": "^9.0.0", + "@nestjs/terminus": "^9.1.2", + "@willsoto/nestjs-prometheus": "^4.7.0", + "ethers": "^5.7.1", + "nest-winston": "^1.7.0", + "prom-client": "^14.1.0", + "reflect-metadata": "^0.1.13", + "rimraf": "^3.0.2", + "rxjs": "^7.2.0", + "winston": "^3.8.2", + "zksync-web3": "0.15.4" + }, + "devDependencies": { + "@nestjs/cli": "^9.0.0", + "@nestjs/schematics": "^9.0.0", + "@nestjs/testing": "^9.0.0", + "@types/express": "^4.17.13", + "@types/jest": "28.1.8", + "@types/supertest": "^2.0.11", + "@typescript-eslint/eslint-plugin": "^5.0.0", + "@typescript-eslint/parser": "^5.0.0", + "eslint-config-prettier": "^8.3.0", + "eslint-plugin-prettier": "^4.0.0", + "jest": "29.2.1", + "jest-junit": "^14.0.1", + "jest-mock-extended": "^3.0.1", + "lint-staged": "^13.0.3", + "source-map-support": "^0.5.20", + "supertest": "^6.1.3", + "ts-jest": "29.0.3", + "ts-loader": "^9.2.3", + "ts-node": "^10.0.0", + "tsconfig-paths": "4.1.0" + }, + "engines": { + "node": ">=18.0.0", + "npm": ">=9.0.0" + } + }, + "packages/data-fetcher/node_modules/@ethersproject/networks": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/@ethersproject/networks/-/networks-5.7.1.tgz", + "integrity": "sha512-n/MufjFYv3yFcUyfhnXotyDlNdFb7onmkSy8aQERi2PjNcnWQ66xXxa3XlS8nCcA8aJKJjIIMNJTC7tu80GwpQ==", + "funding": [ + { + "type": "individual", + "url": "https://gitcoin.co/grants/13/ethersjs-complete-simple-and-tiny-2" + }, + { + "type": "individual", + "url": "https://www.buymeacoffee.com/ricmoo" + } + ], + "dependencies": { + "@ethersproject/logger": "^5.7.0" + } + }, + "packages/data-fetcher/node_modules/@ethersproject/providers": { + "version": "5.7.2", + "resolved": "https://registry.npmjs.org/@ethersproject/providers/-/providers-5.7.2.tgz", + "integrity": "sha512-g34EWZ1WWAVgr4aptGlVBF8mhl3VWjv+8hoAnzStu8Ah22VHBsuGzP17eb6xDVRzw895G4W7vvx60lFFur/1Rg==", + "funding": [ + { + "type": "individual", + "url": "https://gitcoin.co/grants/13/ethersjs-complete-simple-and-tiny-2" + }, + { + "type": "individual", + "url": "https://www.buymeacoffee.com/ricmoo" + } + ], + "dependencies": { + "@ethersproject/abstract-provider": "^5.7.0", + "@ethersproject/abstract-signer": "^5.7.0", + "@ethersproject/address": "^5.7.0", + "@ethersproject/base64": "^5.7.0", + "@ethersproject/basex": "^5.7.0", + "@ethersproject/bignumber": "^5.7.0", + "@ethersproject/bytes": "^5.7.0", + "@ethersproject/constants": "^5.7.0", + "@ethersproject/hash": "^5.7.0", + "@ethersproject/logger": "^5.7.0", + "@ethersproject/networks": "^5.7.0", + "@ethersproject/properties": "^5.7.0", + "@ethersproject/random": "^5.7.0", + "@ethersproject/rlp": "^5.7.0", + "@ethersproject/sha2": "^5.7.0", + "@ethersproject/strings": "^5.7.0", + "@ethersproject/transactions": "^5.7.0", + "@ethersproject/web": "^5.7.0", + "bech32": "1.1.4", + "ws": "7.4.6" + } + }, + "packages/data-fetcher/node_modules/@ethersproject/web": { + "version": "5.7.1", + "resolved": "https://registry.npmjs.org/@ethersproject/web/-/web-5.7.1.tgz", + "integrity": "sha512-Gueu8lSvyjBWL4cYsWsjh6MtMwM0+H4HvqFPZfB6dV8ctbP9zFAO73VG1cMWae0FLPCtz0peKPpZY8/ugJJX2w==", + "funding": [ + { + "type": "individual", + "url": "https://gitcoin.co/grants/13/ethersjs-complete-simple-and-tiny-2" + }, + { + "type": "individual", + "url": "https://www.buymeacoffee.com/ricmoo" + } + ], + "dependencies": { + "@ethersproject/base64": "^5.7.0", + "@ethersproject/bytes": "^5.7.0", + "@ethersproject/logger": "^5.7.0", + "@ethersproject/properties": "^5.7.0", + "@ethersproject/strings": "^5.7.0" + } + }, + "packages/data-fetcher/node_modules/ethers": { + "version": "5.7.2", + "resolved": "https://registry.npmjs.org/ethers/-/ethers-5.7.2.tgz", + "integrity": "sha512-wswUsmWo1aOK8rR7DIKiWSw9DbLWe6x98Jrn8wcTflTVvaXhAMaB5zGAXy0GYQEQp9iO1iSHWVyARQm11zUtyg==", + "funding": [ + { + "type": "individual", + "url": "https://gitcoin.co/grants/13/ethersjs-complete-simple-and-tiny-2" + }, + { + "type": "individual", + "url": "https://www.buymeacoffee.com/ricmoo" + } + ], + "dependencies": { + "@ethersproject/abi": "5.7.0", + "@ethersproject/abstract-provider": "5.7.0", + "@ethersproject/abstract-signer": "5.7.0", + "@ethersproject/address": "5.7.0", + "@ethersproject/base64": "5.7.0", + "@ethersproject/basex": "5.7.0", + "@ethersproject/bignumber": "5.7.0", + "@ethersproject/bytes": "5.7.0", + "@ethersproject/constants": "5.7.0", + "@ethersproject/contracts": "5.7.0", + "@ethersproject/hash": "5.7.0", + "@ethersproject/hdnode": "5.7.0", + "@ethersproject/json-wallets": "5.7.0", + "@ethersproject/keccak256": "5.7.0", + "@ethersproject/logger": "5.7.0", + "@ethersproject/networks": "5.7.1", + "@ethersproject/pbkdf2": "5.7.0", + "@ethersproject/properties": "5.7.0", + "@ethersproject/providers": "5.7.2", + "@ethersproject/random": "5.7.0", + "@ethersproject/rlp": "5.7.0", + "@ethersproject/sha2": "5.7.0", + "@ethersproject/signing-key": "5.7.0", + "@ethersproject/solidity": "5.7.0", + "@ethersproject/strings": "5.7.0", + "@ethersproject/transactions": "5.7.0", + "@ethersproject/units": "5.7.0", + "@ethersproject/wallet": "5.7.0", + "@ethersproject/web": "5.7.1", + "@ethersproject/wordlists": "5.7.0" + } + }, + "packages/data-fetcher/node_modules/ws": { + "version": "7.4.6", + "resolved": "https://registry.npmjs.org/ws/-/ws-7.4.6.tgz", + "integrity": "sha512-YmhHDO4MzaDLB+M9ym/mDA5z0naX8j7SIlT8f8z+I0VtzsRbekxEutHSme7NPS2qE8StCYQNUnfWdXta/Yu85A==", + "engines": { + "node": ">=8.3.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": "^5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + }, + "packages/data-fetcher/node_modules/zksync-web3": { + "version": "0.15.4", + "resolved": "https://registry.npmjs.org/zksync-web3/-/zksync-web3-0.15.4.tgz", + "integrity": "sha512-6CEpRBbF4nGwRYSF3KvPGqg2aNJFYTl8AR+cejBnC2Uyu1v3NYSkmkXXVuMGupJ7HIQR1aTqFEDsUFPyO/bL0Q==", + "deprecated": "This package has been deprecated in favor of zksync-ethers@5.0.0", + "peerDependencies": { + "ethers": "^5.7.0" + } + }, "packages/worker": { "version": "0.0.0", "license": "MIT", @@ -55182,6 +55385,7 @@ "rxjs": "^7.2.0", "typeorm": "^0.3.15", "winston": "^3.8.2", + "yargs": "^17.7.2", "zksync-web3": "0.15.4" }, "devDependencies": { @@ -55191,6 +55395,7 @@ "@types/express": "^4.17.13", "@types/jest": "28.1.8", "@types/supertest": "^2.0.11", + "@types/yargs": "^17.0.32", "@typescript-eslint/eslint-plugin": "^5.0.0", "@typescript-eslint/parser": "^5.0.0", "eslint-config-prettier": "^8.3.0", diff --git a/packages/api/.env.example b/packages/api/.env.example index ed85f208b3..89dfe17683 100644 --- a/packages/api/.env.example +++ b/packages/api/.env.example @@ -1,3 +1,4 @@ +GRACEFUL_SHUTDOWN_TIMEOUT_MS=0 DATABASE_URL=postgres://postgres:postgres@localhost:5432/block-explorer DATABASE_REPLICA_URL_0= DATABASE_CONNECTION_POOL_SIZE=50 @@ -13,4 +14,11 @@ DISABLE_BFF_API_SCHEMA_DOCS=false DISABLE_EXTERNAL_API=false DATABASE_STATEMENT_TIMEOUT_MS=90000 CONTRACT_VERIFICATION_API_URL=http://127.0.0.1:3070 -NETWORK_NAME=testnet-goerli +NETWORK_NAME=testnet-sepolia +BASE_TOKEN_SYMBOL=wETH +BASE_TOKEN_DECIMALS=18 +BASE_TOKEN_L1_ADDRESS=0x8E9C82509488eD471A83824d20Dd474b8F534a0b +BASE_TOKEN_ICON_URL=https://assets.coingecko.com/coins/images/279/large/ethereum.png?1698873266 +BASE_TOKEN_NAME=Ether +BASE_TOKEN_LIQUIDITY=220000000000 +BASE_TOKEN_USDPRICE=1800 diff --git a/packages/api/src/address/address.controller.spec.ts b/packages/api/src/address/address.controller.spec.ts index e215f6c378..35586a0f8a 100644 --- a/packages/api/src/address/address.controller.spec.ts +++ b/packages/api/src/address/address.controller.spec.ts @@ -12,7 +12,7 @@ import { Token } from "../token/token.entity"; import { PagingOptionsWithMaxItemsLimitDto } from "../common/dtos"; import { AddressType } from "./dtos/baseAddress.dto"; import { TransferService } from "../transfer/transfer.service"; -import { Transfer } from "../transfer/transfer.entity"; +import { Transfer, TransferType } from "../transfer/transfer.entity"; jest.mock("../common/utils", () => ({ ...jest.requireActual("../common/utils"), @@ -286,8 +286,8 @@ describe("AddressController", () => { (transferServiceMock.findAll as jest.Mock).mockResolvedValueOnce(transfers); }); - it("queries transfers with the specified options", async () => { - await controller.getAddressTransfers(address, listFilterOptions, pagingOptions); + it("queries transfers with the specified options when no filters provided", async () => { + await controller.getAddressTransfers(address, {}, listFilterOptions, pagingOptions); expect(transferServiceMock.findAll).toHaveBeenCalledTimes(1); expect(transferServiceMock.findAll).toHaveBeenCalledWith( { @@ -303,8 +303,25 @@ describe("AddressController", () => { ); }); + it("queries transfers with the specified options when filters are provided", async () => { + await controller.getAddressTransfers(address, { type: TransferType.Transfer }, listFilterOptions, pagingOptions); + expect(transferServiceMock.findAll).toHaveBeenCalledTimes(1); + expect(transferServiceMock.findAll).toHaveBeenCalledWith( + { + address, + type: TransferType.Transfer, + timestamp: "timestamp", + }, + { + filterOptions: { type: TransferType.Transfer, ...listFilterOptions }, + ...pagingOptions, + route: `address/${address}/transfers`, + } + ); + }); + it("returns the transfers", async () => { - const result = await controller.getAddressTransfers(address, listFilterOptions, pagingOptions); + const result = await controller.getAddressTransfers(address, {}, listFilterOptions, pagingOptions); expect(result).toBe(transfers); }); }); diff --git a/packages/api/src/address/address.controller.ts b/packages/api/src/address/address.controller.ts index 70f18a078c..015cb9280c 100644 --- a/packages/api/src/address/address.controller.ts +++ b/packages/api/src/address/address.controller.ts @@ -17,7 +17,7 @@ import { AddressService } from "./address.service"; import { BlockService } from "../block/block.service"; import { TransactionService } from "../transaction/transaction.service"; import { BalanceService } from "../balance/balance.service"; -import { AddressType, ContractDto, AccountDto, TokenAddressDto } from "./dtos"; +import { AddressType, ContractDto, AccountDto, TokenAddressDto, FilterAddressTransfersOptionsDto } from "./dtos"; import { LogDto } from "../log/log.dto"; import { LogService } from "../log/log.service"; import { ParseAddressPipe, ADDRESS_REGEX_PATTERN } from "../common/pipes/parseAddress.pipe"; @@ -140,19 +140,26 @@ export class AddressController { }) public async getAddressTransfers( @Param("address", new ParseAddressPipe()) address: string, + @Query() filterAddressTransferOptions: FilterAddressTransfersOptionsDto, @Query() listFilterOptions: ListFiltersDto, @Query() pagingOptions: PagingOptionsWithMaxItemsLimitDto ): Promise> { - const filterTransactionsListOptions = buildDateFilter(listFilterOptions.fromDate, listFilterOptions.toDate); + const filterTransfersListOptions = buildDateFilter(listFilterOptions.fromDate, listFilterOptions.toDate); return await this.transferService.findAll( { address, - isFeeOrRefund: false, - ...filterTransactionsListOptions, + ...filterTransfersListOptions, + ...(filterAddressTransferOptions.type + ? { + type: filterAddressTransferOptions.type, + } + : { + isFeeOrRefund: false, + }), }, { - filterOptions: listFilterOptions, + filterOptions: { ...filterAddressTransferOptions, ...listFilterOptions }, ...pagingOptions, route: `${entityName}/${address}/transfers`, } diff --git a/packages/api/src/address/dtos/filterAddressTransfersOptions.dto.ts b/packages/api/src/address/dtos/filterAddressTransfersOptions.dto.ts new file mode 100644 index 0000000000..c74c33c53b --- /dev/null +++ b/packages/api/src/address/dtos/filterAddressTransfersOptions.dto.ts @@ -0,0 +1,13 @@ +import { ApiPropertyOptional } from "@nestjs/swagger"; +import { IsOptional } from "class-validator"; +import { TransferType } from "../../transfer/transfer.entity"; + +export class FilterAddressTransfersOptionsDto { + @ApiPropertyOptional({ + description: "Transfer type to filter transfers by", + example: TransferType.Transfer, + enum: TransferType, + }) + @IsOptional() + public readonly type?: TransferType; +} diff --git a/packages/api/src/address/dtos/index.ts b/packages/api/src/address/dtos/index.ts index 9d9d6027c2..cbd3966dd4 100644 --- a/packages/api/src/address/dtos/index.ts +++ b/packages/api/src/address/dtos/index.ts @@ -1,3 +1,4 @@ export * from "./account.dto"; export * from "./baseAddress.dto"; export * from "./contract.dto"; +export * from "./filterAddressTransfersOptions.dto"; diff --git a/packages/api/src/api/account/account.controller.spec.ts b/packages/api/src/api/account/account.controller.spec.ts index f5b2f14c6e..8614220aa1 100644 --- a/packages/api/src/api/account/account.controller.spec.ts +++ b/packages/api/src/api/account/account.controller.spec.ts @@ -1,7 +1,7 @@ import { Test } from "@nestjs/testing"; import { mock } from "jest-mock-extended"; import { BadRequestException, Logger } from "@nestjs/common"; -import { L2_ETH_TOKEN_ADDRESS } from "../../common/constants"; +import { BASE_TOKEN_L2_ADDRESS } from "../../common/constants"; import { BlockService } from "../../block/block.service"; import { BlockDetails } from "../../block/blockDetails.entity"; import { TransactionService } from "../../transaction/transaction.service"; @@ -557,7 +557,7 @@ describe("AccountController", () => { describe("getAccountEtherBalance", () => { it("calls balanceService.getBalance and returns account ether balance", async () => { const response = await controller.getAccountEtherBalance(address); - expect(balanceServiceMock.getBalance).toBeCalledWith(address, L2_ETH_TOKEN_ADDRESS); + expect(balanceServiceMock.getBalance).toBeCalledWith(address, BASE_TOKEN_L2_ADDRESS); expect(response).toEqual({ status: ResponseStatus.OK, message: ResponseMessage.OK, @@ -588,7 +588,7 @@ describe("AccountController", () => { it("calls balanceService.getBalancesByAddresses and returns accounts ether balances", async () => { const response = await controller.getAccountsEtherBalances([address, "address2"]); - expect(balanceServiceMock.getBalancesByAddresses).toBeCalledWith([address, "address2"], L2_ETH_TOKEN_ADDRESS); + expect(balanceServiceMock.getBalancesByAddresses).toBeCalledWith([address, "address2"], BASE_TOKEN_L2_ADDRESS); expect(response).toEqual({ status: ResponseStatus.OK, message: ResponseMessage.OK, diff --git a/packages/api/src/api/account/account.controller.ts b/packages/api/src/api/account/account.controller.ts index 5299fa44b6..3a7608faed 100644 --- a/packages/api/src/api/account/account.controller.ts +++ b/packages/api/src/api/account/account.controller.ts @@ -1,6 +1,6 @@ import { Controller, Get, Query, Logger, UseFilters, ParseArrayPipe, BadRequestException } from "@nestjs/common"; import { ApiTags, ApiExcludeController } from "@nestjs/swagger"; -import { L2_ETH_TOKEN_ADDRESS } from "../../common/constants"; +import { BASE_TOKEN_L2_ADDRESS } from "../../common/constants"; import { TokenType } from "../../token/token.entity"; import { dateToTimestamp } from "../../common/utils"; import { BlockService } from "../../block/block.service"; @@ -176,7 +176,7 @@ export class AccountController { public async getAccountEtherBalance( @Query("address", new ParseAddressPipe()) address: string ): Promise { - const balance = await this.balanceService.getBalance(address, L2_ETH_TOKEN_ADDRESS); + const balance = await this.balanceService.getBalance(address, BASE_TOKEN_L2_ADDRESS); return { status: ResponseStatus.OK, message: ResponseMessage.OK, @@ -201,7 +201,7 @@ export class AccountController { if (uniqueAddresses.length > 20) { throw new BadRequestException("Maximum 20 addresses per request"); } - const balances = await this.balanceService.getBalancesByAddresses(addresses, L2_ETH_TOKEN_ADDRESS); + const balances = await this.balanceService.getBalancesByAddresses(addresses, BASE_TOKEN_L2_ADDRESS); const result = addresses.map((address) => ({ account: address, balance: balances.find((balance) => balance.address.toLowerCase() === address.toLowerCase())?.balance || "0", diff --git a/packages/api/src/api/mappers/internalTransactionMapper.spec.ts b/packages/api/src/api/mappers/internalTransactionMapper.spec.ts index ea81b68cfd..fc21c02074 100644 --- a/packages/api/src/api/mappers/internalTransactionMapper.spec.ts +++ b/packages/api/src/api/mappers/internalTransactionMapper.spec.ts @@ -1,6 +1,6 @@ import { Transfer } from "../../transfer/transfer.entity"; import { TransactionStatus } from "../../transaction/entities/transaction.entity"; -import { L2_ETH_TOKEN_ADDRESS } from "../../common/constants"; +import { BASE_TOKEN_L2_ADDRESS } from "../../common/constants"; import { mapInternalTransactionListItem } from "./internalTransactionMapper"; describe("internalTransactionMapper", () => { @@ -11,7 +11,7 @@ describe("internalTransactionMapper", () => { from: "0xc7e0220d02d549c4846A6EC31D89C3B670Ebe35C", to: "0xc7e0220d02d549c4846A6EC31D89C3B670Ebe35D", amount: "1000000", - tokenAddress: L2_ETH_TOKEN_ADDRESS, + tokenAddress: BASE_TOKEN_L2_ADDRESS, transaction: { blockNumber: 20, receivedAt: new Date("2023-01-01"), diff --git a/packages/api/src/api/stats/stats.controller.spec.ts b/packages/api/src/api/stats/stats.controller.spec.ts index f9851dde0b..18e88a9815 100644 --- a/packages/api/src/api/stats/stats.controller.spec.ts +++ b/packages/api/src/api/stats/stats.controller.spec.ts @@ -2,8 +2,9 @@ import { Test } from "@nestjs/testing"; import { mock } from "jest-mock-extended"; import { Logger } from "@nestjs/common"; import { TokenService } from "../../token/token.service"; -import { Token, ETH_TOKEN } from "../../token/token.entity"; +import { Token } from "../../token/token.entity"; import { StatsController } from "./stats.controller"; +import { baseTokenData } from "../../config"; describe("StatsController", () => { let controller: StatsController; @@ -31,7 +32,7 @@ describe("StatsController", () => { describe("ethPrice", () => { it("returns ok response and ETH price when ETH token is found", async () => { jest.spyOn(tokenServiceMock, "findOne").mockResolvedValueOnce({ - usdPrice: ETH_TOKEN.usdPrice, + usdPrice: baseTokenData.usdPrice, offChainDataUpdatedAt: new Date("2023-03-03"), } as Token); @@ -40,7 +41,7 @@ describe("StatsController", () => { status: "1", message: "OK", result: { - ethusd: ETH_TOKEN.usdPrice.toString(), + ethusd: baseTokenData.usdPrice.toString(), ethusd_timestamp: Math.floor(new Date("2023-03-03").getTime() / 1000).toString(), }, }); diff --git a/packages/api/src/api/stats/stats.controller.ts b/packages/api/src/api/stats/stats.controller.ts index 972edb3680..04618357a5 100644 --- a/packages/api/src/api/stats/stats.controller.ts +++ b/packages/api/src/api/stats/stats.controller.ts @@ -4,8 +4,8 @@ import { ResponseStatus, ResponseMessage } from "../dtos/common/responseBase.dto import { ApiExceptionFilter } from "../exceptionFilter"; import { EthPriceResponseDto } from "../dtos/stats/ethPrice.dto"; import { TokenService } from "../../token/token.service"; -import { ETH_TOKEN } from "../../token/token.entity"; import { dateToTimestamp } from "../../common/utils"; +import { baseTokenData } from "../../config"; const entityName = "stats"; @@ -18,7 +18,10 @@ export class StatsController { @Get("/ethprice") public async ethPrice(): Promise { - const token = await this.tokenService.findOne(ETH_TOKEN.l2Address, { usdPrice: true, offChainDataUpdatedAt: true }); + const token = await this.tokenService.findOne(baseTokenData.l2Address, { + usdPrice: true, + offChainDataUpdatedAt: true, + }); return { status: token ? ResponseStatus.OK : ResponseStatus.NOTOK, message: token ? ResponseMessage.OK : ResponseMessage.NO_DATA_FOUND, diff --git a/packages/api/src/api/token/token.controller.spec.ts b/packages/api/src/api/token/token.controller.spec.ts index e3d44eee71..b1b2b38919 100644 --- a/packages/api/src/api/token/token.controller.spec.ts +++ b/packages/api/src/api/token/token.controller.spec.ts @@ -2,9 +2,10 @@ import { Test } from "@nestjs/testing"; import { mock } from "jest-mock-extended"; import { Logger } from "@nestjs/common"; import { TokenService } from "../../token/token.service"; -import { Token, ETH_TOKEN } from "../../token/token.entity"; +import { Token } from "../../token/token.entity"; import { TokenController } from "./token.controller"; - +import config from "../../config/index"; +const { baseTokenData } = config(); describe("TokenController", () => { let controller: TokenController; let tokenServiceMock: TokenService; @@ -32,22 +33,22 @@ describe("TokenController", () => { describe("tokenInfo", () => { it("returns ok response and token info when token is found", async () => { - jest.spyOn(tokenServiceMock, "findOne").mockResolvedValueOnce(ETH_TOKEN); - + const baseToken = baseTokenData as Token; + jest.spyOn(tokenServiceMock, "findOne").mockResolvedValueOnce(baseToken); const response = await controller.tokenInfo(contractAddress); expect(response).toEqual({ status: "1", message: "OK", result: [ { - contractAddress: ETH_TOKEN.l2Address, - iconURL: ETH_TOKEN.iconURL, - l1Address: ETH_TOKEN.l1Address, - liquidity: ETH_TOKEN.liquidity.toString(), - symbol: ETH_TOKEN.symbol, - tokenDecimal: ETH_TOKEN.decimals.toString(), - tokenName: ETH_TOKEN.name, - tokenPriceUSD: ETH_TOKEN.usdPrice.toString(), + contractAddress: baseToken.l2Address, + iconURL: baseToken.iconURL, + l1Address: baseToken.l1Address, + liquidity: baseToken.liquidity.toString(), + symbol: baseToken.symbol, + tokenDecimal: baseToken.decimals.toString(), + tokenName: baseToken.name, + tokenPriceUSD: baseToken.usdPrice.toString(), }, ], }); diff --git a/packages/api/src/api/token/token.controller.ts b/packages/api/src/api/token/token.controller.ts index 95c559696c..caa9ce8d30 100644 --- a/packages/api/src/api/token/token.controller.ts +++ b/packages/api/src/api/token/token.controller.ts @@ -5,7 +5,6 @@ import { ResponseStatus, ResponseMessage } from "../dtos/common/responseBase.dto import { ApiExceptionFilter } from "../exceptionFilter"; import { TokenInfoResponseDto } from "../dtos/token/tokenInfo.dto"; import { TokenService } from "../../token/token.service"; - const entityName = "token"; @ApiExcludeController() diff --git a/packages/api/src/balance/balance.entity.ts b/packages/api/src/balance/balance.entity.ts index 1d0dc898e5..34e3f7544c 100644 --- a/packages/api/src/balance/balance.entity.ts +++ b/packages/api/src/balance/balance.entity.ts @@ -1,9 +1,9 @@ import { Entity, Column, PrimaryColumn, Index, ManyToOne, JoinColumn, AfterLoad } from "typeorm"; import { BaseEntity } from "../common/entities/base.entity"; -import { Token, ETH_TOKEN } from "../token/token.entity"; +import { Token } from "../token/token.entity"; import { normalizeAddressTransformer } from "../common/transformers/normalizeAddress.transformer"; import { bigIntNumberTransformer } from "../common/transformers/bigIntNumber.transformer"; - +import { baseTokenData } from "../config/index"; @Entity({ name: "balances" }) export class Balance extends BaseEntity { @PrimaryColumn({ type: "bytea", transformer: normalizeAddressTransformer }) @@ -24,9 +24,12 @@ export class Balance extends BaseEntity { public readonly balance: string; @AfterLoad() - populateEthToken() { - if (this.tokenAddress === ETH_TOKEN.l2Address && !this.token) { - this.token = ETH_TOKEN; + populateBaseToken() { + if ( + !this.token && + (this.tokenAddress === undefined || this.tokenAddress.toLowerCase() === baseTokenData.l2Address.toLowerCase()) + ) { + this.token = baseTokenData as Token; } } } diff --git a/packages/api/src/common/constants.ts b/packages/api/src/common/constants.ts index 89ed3cf93b..99cc0c6649 100644 --- a/packages/api/src/common/constants.ts +++ b/packages/api/src/common/constants.ts @@ -1 +1 @@ -export const L2_ETH_TOKEN_ADDRESS = "0x000000000000000000000000000000000000800a"; +export const BASE_TOKEN_L2_ADDRESS = "0x000000000000000000000000000000000000800A"; diff --git a/packages/api/src/common/types.ts b/packages/api/src/common/types.ts index 743cd580ca..4977f91f51 100644 --- a/packages/api/src/common/types.ts +++ b/packages/api/src/common/types.ts @@ -1,4 +1,5 @@ import { IPaginationOptions as NestIPaginationOptions, IPaginationMeta } from "nestjs-typeorm-paginate"; +import { TransferType } from "../transfer/transfer.entity"; interface IPaginationFilterOptions { fromDate?: string; @@ -7,6 +8,7 @@ interface IPaginationFilterOptions { address?: string; l1BatchNumber?: number; minLiquidity?: number; + type?: TransferType; } export interface IPaginationOptions extends NestIPaginationOptions { diff --git a/packages/api/src/config/docs/constants.testnet-goerli.json b/packages/api/src/config/docs/constants.testnet-goerli.json deleted file mode 100644 index e848c9d3b5..0000000000 --- a/packages/api/src/config/docs/constants.testnet-goerli.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "verifiedContractAddress": "0x53E185A2FA7c9caF14A887E8E9a4862D4bd094ea", - "verifiedContractAddress2": "0xbf2A1ACE3B12b81bab4985f05E850AcFCCb416E0", - "contractAddressWithLogs": "0xbf2A1ACE3B12b81bab4985f05E850AcFCCb416E0", - "txHash": "0x3d36c6e6a3625d698ef41d20c9457a6628254c8307df54b7c887e30f7dda00c8", - "address": "0xE4ce1da467a7Ca37727eb7e19857e5167DE25966", - "addressWithInternalTx": "0xbf2A1ACE3B12b81bab4985f05E850AcFCCb416E0", - "addressTxWithInternalTransfers": "0x8a453b8dd3e095b3034dc3692663d5bf0c9883cbe6e9f9a0425a3ebf9b9360ab", - "tokenAddress": "0x000000000000000000000000000000000000800A", - "erc20TokenAddress": "0x0faF6df7054946141266420b43783387A78d82A9", - "erc721TokenAddress": "0x09B0196641D91eDEC4042e4bb8C605bb35a02546", - "erc721TokenHolderAddress": "0xFb7E0856e44Eff812A44A9f47733d7d55c39Aa28" -} \ No newline at end of file diff --git a/packages/api/src/config/index.spec.ts b/packages/api/src/config/index.spec.ts index f7a1b7bf14..22bfe07416 100644 --- a/packages/api/src/config/index.spec.ts +++ b/packages/api/src/config/index.spec.ts @@ -1,5 +1,4 @@ import config from "../config"; - jest.mock("./featureFlags", () => ({ feature1Enabled: true, feature2Enabled: false, @@ -20,6 +19,71 @@ describe("config", () => { it("sets default values", () => { expect(config()).toEqual({ + baseTokenData: { + l2Address: "0x000000000000000000000000000000000000800A", + l1Address: "0x0000000000000000000000000000000000000001", + symbol: "ETH", + name: "Ether", + decimals: 18, + // Fallback data in case ETH token is not in the DB + iconURL: "https://assets.coingecko.com/coins/images/279/large/ethereum.png?1698873266", + liquidity: 220000000000, + usdPrice: 1800, + }, + NODE_ENV: "test", + port: 3020, + metrics: { + port: 3005, + collectDbConnectionPoolMetricsInterval: 10000, + }, + typeORM: { + type: "postgres", + url: "postgres://postgres:postgres@127.0.0.1:5432/block-explorer", + poolSize: 300, + extra: { + idleTimeoutMillis: 60000, + statement_timeout: 90000, + }, + synchronize: true, + logging: false, + autoLoadEntities: true, + retryAttempts: 10, + retryDelay: 3000, + applicationName: "block-explorer-api", + }, + contractVerificationApiUrl: "http://127.0.0.1:3070", + featureFlags: { + feature1Enabled: true, + feature2Enabled: false, + }, + gracefulShutdownTimeoutMs: 0, + }); + }); + + it("sets default values with base ERC20", () => { + process.env = { + BASE_TOKEN_SYMBOL: "MTTL", + BASE_TOKEN_DECIMALS: "18", + BASE_TOKEN_L1_ADDRESS: "0xSomeAddress", + BASE_TOKEN_ICON_URL: "https://matter-labs.io", + BASE_TOKEN_NAME: "MatterLabs", + BASE_TOKEN_LIQUIDITY: "999999999999", + BASE_TOKEN_USDPRICE: "19", + NODE_ENV: "test", + }; + + expect(config()).toEqual({ + baseTokenData: { + l2Address: "0x000000000000000000000000000000000000800A", + l1Address: "0xSomeAddress", + symbol: "MTTL", + name: "MatterLabs", + decimals: 18, + // Fallback data in case ETH token is not in the DB + iconURL: "https://matter-labs.io", + liquidity: 999999999999, + usdPrice: 19, + }, NODE_ENV: "test", port: 3020, metrics: { @@ -28,7 +92,7 @@ describe("config", () => { }, typeORM: { type: "postgres", - url: "postgres://postgres:postgres@localhost:5432/block-explorer", + url: "postgres://postgres:postgres@127.0.0.1:5432/block-explorer", poolSize: 300, extra: { idleTimeoutMillis: 60000, @@ -46,6 +110,7 @@ describe("config", () => { feature1Enabled: true, feature2Enabled: false, }, + gracefulShutdownTimeoutMs: 0, }); }); diff --git a/packages/api/src/config/index.ts b/packages/api/src/config/index.ts index 6b9c5f2a57..41077cd39b 100644 --- a/packages/api/src/config/index.ts +++ b/packages/api/src/config/index.ts @@ -1,5 +1,55 @@ import { TypeOrmModuleOptions } from "@nestjs/typeorm"; import * as featureFlags from "./featureFlags"; +import { BASE_TOKEN_L2_ADDRESS } from "../common/constants"; +type BaseToken = { + symbol: string; + decimals: number; + l1Address: string; + l2Address: string; + liquidity: number; + iconURL: string; + name: string; + usdPrice: number; +}; +const defaultEthBaseToken: BaseToken = { + l2Address: BASE_TOKEN_L2_ADDRESS, + l1Address: "0x0000000000000000000000000000000000000001", + symbol: "ETH", + name: "Ether", + decimals: 18, + // Fallback data in case ETH token is not in the DB + iconURL: "https://assets.coingecko.com/coins/images/279/large/ethereum.png?1698873266", + liquidity: 220000000000, + usdPrice: 1800, +}; +const baseTokenFromEnv = (): BaseToken => { + const { + BASE_TOKEN_SYMBOL, + BASE_TOKEN_DECIMALS, + BASE_TOKEN_L1_ADDRESS, + BASE_TOKEN_ICON_URL, + BASE_TOKEN_NAME, + BASE_TOKEN_LIQUIDITY, + BASE_TOKEN_USDPRICE, + } = process.env; + const decimals = parseFloat(BASE_TOKEN_DECIMALS); + const liquidity = parseFloat(BASE_TOKEN_LIQUIDITY); + const usdPrice = parseFloat(BASE_TOKEN_USDPRICE); + if (BASE_TOKEN_L1_ADDRESS && BASE_TOKEN_SYMBOL) { + return { + symbol: BASE_TOKEN_SYMBOL, + decimals, + l1Address: BASE_TOKEN_L1_ADDRESS, + l2Address: BASE_TOKEN_L2_ADDRESS, + liquidity, + iconURL: BASE_TOKEN_ICON_URL, + usdPrice, + name: BASE_TOKEN_NAME, + }; + } else { + return defaultEthBaseToken; + } +}; export default () => { const { @@ -12,8 +62,11 @@ export default () => { DATABASE_CONNECTION_IDLE_TIMEOUT_MS, DATABASE_STATEMENT_TIMEOUT_MS, CONTRACT_VERIFICATION_API_URL, + GRACEFUL_SHUTDOWN_TIMEOUT_MS, } = process.env; + const baseTokenData: BaseToken = baseTokenFromEnv(); + const MAX_NUMBER_OF_REPLICA = 100; const getDatabaseReplicaSet = () => { @@ -31,7 +84,7 @@ export default () => { }; const getTypeOrmModuleOptions = (): TypeOrmModuleOptions => { - const master = { url: DATABASE_URL || "postgres://postgres:postgres@localhost:5432/block-explorer" }; + const master = { url: DATABASE_URL || "postgres://postgres:postgres@127.0.0.1:5432/block-explorer" }; const replicaSet = getDatabaseReplicaSet(); return { @@ -74,5 +127,9 @@ export default () => { typeORM: getTypeOrmModuleOptions(), contractVerificationApiUrl: CONTRACT_VERIFICATION_API_URL || "http://127.0.0.1:3070", featureFlags, + baseTokenData, + gracefulShutdownTimeoutMs: parseInt(GRACEFUL_SHUTDOWN_TIMEOUT_MS, 10) || 0, }; }; + +export const baseTokenData = baseTokenFromEnv(); diff --git a/packages/api/src/health/health.controller.spec.ts b/packages/api/src/health/health.controller.spec.ts index 54770ddda1..9bfd136f56 100644 --- a/packages/api/src/health/health.controller.spec.ts +++ b/packages/api/src/health/health.controller.spec.ts @@ -1,14 +1,25 @@ +import { ServiceUnavailableException } from "@nestjs/common"; import { Test, TestingModule } from "@nestjs/testing"; import { HealthCheckService, TypeOrmHealthIndicator, HealthCheckResult } from "@nestjs/terminus"; import { mock } from "jest-mock-extended"; +import { ConfigService } from "@nestjs/config"; +import { setTimeout } from "node:timers/promises"; import { HealthController } from "./health.controller"; +jest.mock("node:timers/promises", () => ({ + setTimeout: jest.fn().mockResolvedValue(null), +})); + describe("HealthController", () => { let healthCheckServiceMock: HealthCheckService; let dbHealthCheckerMock: TypeOrmHealthIndicator; + let configServiceMock: ConfigService; let healthController: HealthController; beforeEach(async () => { + configServiceMock = mock({ + get: jest.fn().mockReturnValue(1), + }); healthCheckServiceMock = mock({ check: jest.fn().mockImplementation((healthChecks) => { for (const healthCheck of healthChecks) { @@ -30,6 +41,10 @@ describe("HealthController", () => { provide: TypeOrmHealthIndicator, useValue: dbHealthCheckerMock, }, + { + provide: ConfigService, + useValue: configServiceMock, + }, ], }).compile(); @@ -49,5 +64,54 @@ describe("HealthController", () => { const result = await healthController.check(); expect(result).toBe(healthCheckResult); }); + + describe("when health checks fail with an error", () => { + const error: ServiceUnavailableException = new ServiceUnavailableException({ + status: "error", + db: { + status: "down", + }, + }); + + beforeEach(() => { + jest.spyOn(healthCheckServiceMock, "check").mockImplementation(() => { + throw error; + }); + }); + + it("throws generated error", async () => { + expect.assertions(4); + try { + await healthController.check(); + } catch (e) { + expect(e).toBeInstanceOf(ServiceUnavailableException); + expect(e.message).toBe("Service Unavailable Exception"); + expect(e.response).toEqual(error.getResponse()); + expect(e.stack).toEqual(error.stack); + } + }); + }); + }); + + describe("beforeApplicationShutdown", () => { + beforeEach(() => { + (setTimeout as jest.Mock).mockReset(); + }); + + it("defined and returns void", async () => { + const result = await healthController.beforeApplicationShutdown(); + expect(result).toBeUndefined(); + }); + + it("awaits configured shutdown timeout", async () => { + await healthController.beforeApplicationShutdown("SIGTERM"); + expect(setTimeout).toBeCalledTimes(1); + expect(setTimeout).toBeCalledWith(1); + }); + + it("does not await shutdown timeout if signal is not SIGTERM", async () => { + await healthController.beforeApplicationShutdown("SIGINT"); + expect(setTimeout).toBeCalledTimes(0); + }); }); }); diff --git a/packages/api/src/health/health.controller.ts b/packages/api/src/health/health.controller.ts index d5b5e7bf8b..a3b3509fe1 100644 --- a/packages/api/src/health/health.controller.ts +++ b/packages/api/src/health/health.controller.ts @@ -1,18 +1,40 @@ -import { Controller, Get } from "@nestjs/common"; +import { Logger, Controller, Get, BeforeApplicationShutdown } from "@nestjs/common"; import { HealthCheckService, TypeOrmHealthIndicator, HealthCheck, HealthCheckResult } from "@nestjs/terminus"; import { ApiExcludeController } from "@nestjs/swagger"; +import { ConfigService } from "@nestjs/config"; +import { setTimeout } from "node:timers/promises"; @ApiExcludeController() @Controller(["health", "ready"]) -export class HealthController { +export class HealthController implements BeforeApplicationShutdown { + private readonly logger: Logger; + private readonly gracefulShutdownTimeoutMs: number; + constructor( private readonly healthCheckService: HealthCheckService, - private readonly dbHealthChecker: TypeOrmHealthIndicator - ) {} + private readonly dbHealthChecker: TypeOrmHealthIndicator, + configService: ConfigService + ) { + this.logger = new Logger(HealthController.name); + this.gracefulShutdownTimeoutMs = configService.get("gracefulShutdownTimeoutMs"); + } @Get() @HealthCheck() public async check(): Promise { - return await this.healthCheckService.check([() => this.dbHealthChecker.pingCheck("database")]); + try { + return await this.healthCheckService.check([() => this.dbHealthChecker.pingCheck("database")]); + } catch (error) { + this.logger.error({ message: error.message, response: error.getResponse() }, error.stack); + throw error; + } + } + + public async beforeApplicationShutdown(signal?: string): Promise { + if (this.gracefulShutdownTimeoutMs && signal === "SIGTERM") { + this.logger.debug(`Awaiting ${this.gracefulShutdownTimeoutMs}ms before shutdown`); + await setTimeout(this.gracefulShutdownTimeoutMs); + this.logger.debug(`Timeout reached, shutting down now`); + } } } diff --git a/packages/api/src/logger.ts b/packages/api/src/logger.ts index 8a53479c6a..d9c1fc7875 100644 --- a/packages/api/src/logger.ts +++ b/packages/api/src/logger.ts @@ -5,9 +5,11 @@ import { format, transports, Logform } from "winston"; export const getLogger = (environment: string, logLevel: string): LoggerService => { let defaultLogLevel = "debug"; const loggerFormatters: Logform.Format[] = [ - format.timestamp({ - format: "DD/MM/YYYY HH:mm:ss.SSS", - }), + environment === "production" + ? format.timestamp() + : format.timestamp({ + format: "DD/MM/YYYY HH:mm:ss.SSS", + }), format.ms(), utilities.format.nestLike("API", {}), ]; diff --git a/packages/api/src/main.ts b/packages/api/src/main.ts index 49246daeb9..68d64832cb 100644 --- a/packages/api/src/main.ts +++ b/packages/api/src/main.ts @@ -2,11 +2,14 @@ import helmet from "helmet"; import { NestFactory } from "@nestjs/core"; import { SwaggerModule, DocumentBuilder } from "@nestjs/swagger"; import { ConfigService } from "@nestjs/config"; +import { NestExpressApplication } from "@nestjs/platform-express"; import { configureApp } from "./configureApp"; import { getLogger } from "./logger"; import { AppModule } from "./app.module"; import { AppMetricsModule } from "./appMetrics.module"; +const BODY_PARSER_SIZE_LIMIT = "10mb"; + async function bootstrap() { const logger = getLogger(process.env.NODE_ENV, process.env.LOG_LEVEL); @@ -15,8 +18,9 @@ async function bootstrap() { process.exit(1); }); - const app = await NestFactory.create(AppModule, { + const app = await NestFactory.create(AppModule, { logger, + rawBody: true, }); const configService = app.get(ConfigService); const metricsApp = await NestFactory.create(AppMetricsModule); @@ -32,6 +36,8 @@ async function bootstrap() { SwaggerModule.setup("docs", app, document); } + app.useBodyParser("json", { limit: BODY_PARSER_SIZE_LIMIT }); + app.useBodyParser("urlencoded", { limit: BODY_PARSER_SIZE_LIMIT, extended: true }); app.enableCors(); app.use(helmet()); configureApp(app); diff --git a/packages/api/src/token/token.entity.ts b/packages/api/src/token/token.entity.ts index 234bc60315..f8c563bbe5 100644 --- a/packages/api/src/token/token.entity.ts +++ b/packages/api/src/token/token.entity.ts @@ -1,25 +1,12 @@ import { Entity, Column, PrimaryColumn, Index } from "typeorm"; import { BaseEntity } from "../common/entities/base.entity"; import { normalizeAddressTransformer } from "../common/transformers/normalizeAddress.transformer"; - export enum TokenType { - ETH = "ETH", + BaseToken = "BASETOKEN", ERC20 = "ERC20", ERC721 = "ERC721", } -export const ETH_TOKEN: Token = { - l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", - symbol: "ETH", - name: "Ether", - decimals: 18, - // Fallback data in case ETH token is not in the DB - iconURL: "https://assets.coingecko.com/coins/images/279/large/ethereum.png?1698873266", - liquidity: 220000000000, - usdPrice: 1800, -} as Token; - @Entity({ name: "tokens" }) @Index(["liquidity", "blockNumber", "logIndex"]) export class Token extends BaseEntity { diff --git a/packages/api/src/token/token.module.ts b/packages/api/src/token/token.module.ts index 5e74dfa05e..43799ffab0 100644 --- a/packages/api/src/token/token.module.ts +++ b/packages/api/src/token/token.module.ts @@ -6,7 +6,6 @@ import { Token } from "./token.entity"; import { Block } from "../block/block.entity"; import { Transaction } from "../transaction/entities/transaction.entity"; import { TransferModule } from "../transfer/transfer.module"; - @Module({ imports: [TypeOrmModule.forFeature([Token, Block, Transaction]), TransferModule], controllers: [TokenController], diff --git a/packages/api/src/token/token.service.spec.ts b/packages/api/src/token/token.service.spec.ts index 2b61764b4f..257d724335 100644 --- a/packages/api/src/token/token.service.spec.ts +++ b/packages/api/src/token/token.service.spec.ts @@ -3,9 +3,11 @@ import { mock } from "jest-mock-extended"; import { getRepositoryToken } from "@nestjs/typeorm"; import { Repository, SelectQueryBuilder, MoreThanOrEqual } from "typeorm"; import { TokenService } from "./token.service"; -import { Token, ETH_TOKEN } from "./token.entity"; +import { Token } from "./token.entity"; import { Pagination, IPaginationMeta } from "nestjs-typeorm-paginate"; import * as utils from "../common/utils"; +import config from "../config"; +const { baseTokenData } = config(); jest.mock("../common/utils"); @@ -73,7 +75,7 @@ describe("TokenService", () => { it("returns ETH token for ETH address", async () => { const result = await service.findOne("0x000000000000000000000000000000000000800a"); - expect(result).toEqual(ETH_TOKEN); + expect(result).toEqual(baseTokenData); }); it("returns null for non ETH address", async () => { diff --git a/packages/api/src/token/token.service.ts b/packages/api/src/token/token.service.ts index 4adafbc315..559606467c 100644 --- a/packages/api/src/token/token.service.ts +++ b/packages/api/src/token/token.service.ts @@ -4,8 +4,10 @@ import { Repository, FindOptionsSelect, MoreThanOrEqual } from "typeorm"; import { Pagination } from "nestjs-typeorm-paginate"; import { IPaginationOptions } from "../common/types"; import { paginate } from "../common/utils"; -import { Token, ETH_TOKEN } from "./token.entity"; - +import { Token } from "./token.entity"; +import { BASE_TOKEN_L2_ADDRESS } from "../common/constants"; +import config from "../config"; +const { baseTokenData } = config(); export interface FilterTokensOptions { minLiquidity?: number; } @@ -24,8 +26,8 @@ export class TokenService { }, select: fields, }); - if (!token && address.toLowerCase() === ETH_TOKEN.l2Address.toLowerCase()) { - return ETH_TOKEN; + if (!token && address.toLowerCase() === BASE_TOKEN_L2_ADDRESS.toLowerCase()) { + return baseTokenData as Token; } return token; } @@ -33,7 +35,7 @@ export class TokenService { public async exists(address: string): Promise { const tokenExists = (await this.tokenRepository.findOne({ where: { l2Address: address }, select: { l2Address: true } })) != null; - if (!tokenExists && address === ETH_TOKEN.l2Address.toLowerCase()) { + if (!tokenExists && address === BASE_TOKEN_L2_ADDRESS.toLowerCase()) { return true; } return tokenExists; diff --git a/packages/api/src/transaction/transaction.controller.ts b/packages/api/src/transaction/transaction.controller.ts index f8465723d9..846bf0f61c 100644 --- a/packages/api/src/transaction/transaction.controller.ts +++ b/packages/api/src/transaction/transaction.controller.ts @@ -101,13 +101,14 @@ export class TransactionController { throw new NotFoundException(); } - return await this.transferService.findAll( + const transfers = await this.transferService.findAll( { transactionHash }, { ...pagingOptions, route: `${entityName}/${transactionHash}/transfers`, } ); + return transfers; } @Get(":transactionHash/logs") diff --git a/packages/api/src/transfer/addressTransfer.entity.ts b/packages/api/src/transfer/addressTransfer.entity.ts index 65e9677020..5b1a9a5a1c 100644 --- a/packages/api/src/transfer/addressTransfer.entity.ts +++ b/packages/api/src/transfer/addressTransfer.entity.ts @@ -1,12 +1,13 @@ import { Entity, Column, Index, ManyToOne, JoinColumn, PrimaryColumn } from "typeorm"; import { BaseEntity } from "../common/entities/base.entity"; -import { Transfer } from "./transfer.entity"; +import { Transfer, TransferType } from "./transfer.entity"; import { TokenType } from "../token/token.entity"; import { bigIntNumberTransformer } from "../common/transformers/bigIntNumber.transformer"; import { normalizeAddressTransformer } from "../common/transformers/normalizeAddress.transformer"; @Entity({ name: "addressTransfers" }) @Index(["address", "isFeeOrRefund", "timestamp", "logIndex"]) +@Index(["address", "type", "timestamp", "logIndex"]) @Index(["address", "tokenType", "blockNumber", "logIndex"]) @Index(["address", "tokenAddress", "blockNumber", "logIndex"]) export class AddressTransfer extends BaseEntity { @@ -34,7 +35,10 @@ export class AddressTransfer extends BaseEntity { @Column({ type: "timestamp" }) public readonly timestamp: Date; - @Column({ type: "enum", enum: TokenType, default: TokenType.ETH }) + @Column({ type: "enum", enum: TransferType, default: TransferType.Transfer }) + public readonly type: TransferType; + + @Column({ type: "enum", enum: TokenType, default: TokenType.BaseToken }) public readonly tokenType: TokenType; @Column({ type: "boolean" }) diff --git a/packages/api/src/transfer/transfer.entity.ts b/packages/api/src/transfer/transfer.entity.ts index c4ec6c6727..3cf5d3e3d0 100644 --- a/packages/api/src/transfer/transfer.entity.ts +++ b/packages/api/src/transfer/transfer.entity.ts @@ -1,10 +1,11 @@ import { Entity, Column, Index, ManyToOne, JoinColumn, PrimaryColumn, AfterLoad } from "typeorm"; import { BaseEntity } from "../common/entities/base.entity"; -import { Token, TokenType, ETH_TOKEN } from "../token/token.entity"; +import { Token, TokenType } from "../token/token.entity"; import { normalizeAddressTransformer } from "../common/transformers/normalizeAddress.transformer"; import { bigIntNumberTransformer } from "../common/transformers/bigIntNumber.transformer"; import { hexTransformer } from "../common/transformers/hex.transformer"; import { Transaction } from "../transaction/entities/transaction.entity"; +import { baseTokenData } from "../config/index"; export enum TransferType { Deposit = "deposit", @@ -64,7 +65,7 @@ export class Transfer extends BaseEntity { @Column({ type: "enum", enum: TransferType, default: TransferType.Transfer }) public readonly type: TransferType; - @Column({ type: "enum", enum: TokenType, default: TokenType.ETH }) + @Column({ type: "enum", enum: TokenType, default: TokenType.BaseToken }) public readonly tokenType: TokenType; @Column({ type: "boolean", select: false }) @@ -86,9 +87,9 @@ export class Transfer extends BaseEntity { } @AfterLoad() - populateEthToken() { - if (!this.token && this.tokenAddress === ETH_TOKEN.l2Address) { - this.token = ETH_TOKEN; + populateBaseToken() { + if (!this.token && this.tokenAddress.toLowerCase() === baseTokenData.l2Address.toLowerCase()) { + this.token = baseTokenData as Token; } } } diff --git a/packages/api/src/transfer/transfer.module.ts b/packages/api/src/transfer/transfer.module.ts index e79e618bad..0b1cdd174c 100644 --- a/packages/api/src/transfer/transfer.module.ts +++ b/packages/api/src/transfer/transfer.module.ts @@ -3,7 +3,6 @@ import { TypeOrmModule } from "@nestjs/typeorm"; import { TransferService } from "./transfer.service"; import { Transfer } from "./transfer.entity"; import { AddressTransfer } from "./addressTransfer.entity"; - @Module({ imports: [TypeOrmModule.forFeature([Transfer, AddressTransfer])], providers: [TransferService], diff --git a/packages/api/src/transfer/transfer.service.ts b/packages/api/src/transfer/transfer.service.ts index 98ce2a2b6b..5491954d99 100644 --- a/packages/api/src/transfer/transfer.service.ts +++ b/packages/api/src/transfer/transfer.service.ts @@ -4,7 +4,7 @@ import { Repository, FindOperator, MoreThanOrEqual, LessThanOrEqual } from "type import { Pagination } from "nestjs-typeorm-paginate"; import { paginate } from "../common/utils"; import { IPaginationOptions, SortingOrder } from "../common/types"; -import { Transfer } from "./transfer.entity"; +import { Transfer, TransferType } from "./transfer.entity"; import { TokenType } from "../token/token.entity"; import { AddressTransfer } from "./addressTransfer.entity"; import { normalizeAddressTransformer } from "../common/transformers/normalizeAddress.transformer"; @@ -15,6 +15,7 @@ export interface FilterTransfersOptions { address?: string; timestamp?: FindOperator; isFeeOrRefund?: boolean; + type?: TransferType; } export interface FilterTokenTransfersOptions { diff --git a/packages/api/test/account-api.e2e-spec.ts b/packages/api/test/account-api.e2e-spec.ts index f529cc56aa..cee6dbd0ea 100644 --- a/packages/api/test/account-api.e2e-spec.ts +++ b/packages/api/test/account-api.e2e-spec.ts @@ -12,7 +12,7 @@ import { Token, TokenType } from "../src/token/token.entity"; import { Balance } from "../src/balance/balance.entity"; import { AddressTransfer } from "../src/transfer/addressTransfer.entity"; import { Transfer, TransferType } from "../src/transfer/transfer.entity"; -import { L2_ETH_TOKEN_ADDRESS } from "../src/common/constants"; +import { BASE_TOKEN_L2_ADDRESS } from "../src/common/constants"; import { AppModule } from "../src/app.module"; import { configureApp } from "../src/configureApp"; @@ -125,7 +125,7 @@ describe("Account API (e2e)", () => { const tokens = [ { - tokenType: TokenType.ETH, + tokenType: TokenType.BaseToken, tokenAddress: "0x000000000000000000000000000000000000800a", }, { @@ -170,14 +170,14 @@ describe("Account API (e2e)", () => { await balanceRepository.insert({ address: "0xc7e0220d02d549c4846A6EC31D89C3B670Ebe35C", - tokenAddress: L2_ETH_TOKEN_ADDRESS, + tokenAddress: BASE_TOKEN_L2_ADDRESS, blockNumber: 1, balance: "1000", }); await balanceRepository.insert({ address: "0xc7e0220d02d549c4846A6EC31D89C3B670Ebe35E", - tokenAddress: L2_ETH_TOKEN_ADDRESS, + tokenAddress: BASE_TOKEN_L2_ADDRESS, blockNumber: 1, balance: "100", }); diff --git a/packages/api/test/address.e2e-spec.ts b/packages/api/test/address.e2e-spec.ts index 0c51412dcc..c47bb1a9e1 100644 --- a/packages/api/test/address.e2e-spec.ts +++ b/packages/api/test/address.e2e-spec.ts @@ -12,13 +12,15 @@ import { Transaction } from "../src/transaction/entities/transaction.entity"; import { AddressTransaction } from "../src/transaction/entities/addressTransaction.entity"; import { TransactionReceipt } from "../src/transaction/entities/transactionReceipt.entity"; import { Log } from "../src/log/log.entity"; -import { Token, TokenType, ETH_TOKEN } from "../src/token/token.entity"; +import { Token, TokenType } from "../src/token/token.entity"; import { BatchDetails } from "../src/batch/batchDetails.entity"; import { Counter } from "../src/counter/counter.entity"; import { Transfer, TransferType } from "../src/transfer/transfer.entity"; import { AddressTransfer } from "../src/transfer/addressTransfer.entity"; +import { baseTokenData } from "../src/config"; describe("AddressController (e2e)", () => { + const ETH_TOKEN = baseTokenData; let app: INestApplication; let addressRepository: Repository
; let blockRepository: Repository; @@ -325,7 +327,7 @@ describe("AddressController (e2e)", () => { transactionIndex: i, timestamp: new Date("2022-11-21T18:16:51.000Z"), type, - tokenType: i % 2 ? TokenType.ERC20 : TokenType.ETH, + tokenType: i % 2 ? TokenType.ERC20 : TokenType.BaseToken, tokenAddress: i % 2 ? "0x97d0a23f34e535e44df8ba84c53a0945cf0eeb67" : "0x000000000000000000000000000000000000800a", logIndex: i, @@ -342,6 +344,7 @@ describe("AddressController (e2e)", () => { tokenAddress: transferSpec.tokenAddress, blockNumber: transferSpec.blockNumber, timestamp: transferSpec.timestamp, + type: transferSpec.type, tokenType: transferSpec.tokenType, isFeeOrRefund: transferSpec.isFeeOrRefund, logIndex: transferSpec.logIndex, @@ -397,7 +400,7 @@ describe("AddressController (e2e)", () => { balance: "34500", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -466,7 +469,7 @@ describe("AddressController (e2e)", () => { balance: "34500", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -535,7 +538,7 @@ describe("AddressController (e2e)", () => { balance: "34500", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -604,7 +607,7 @@ describe("AddressController (e2e)", () => { balance: "34500", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -675,7 +678,7 @@ describe("AddressController (e2e)", () => { balance: "34500", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -1167,6 +1170,22 @@ describe("AddressController (e2e)", () => { ); }); + it("returns HTTP 200 and address transfers for the specified transfer type", () => { + return request(app.getHttpServer()) + .get("/address/0x91d0a23f34e535e44df8ba84c53a0945cf0eeb67/transfers?type=withdrawal") + .expect(200) + .expect((res) => + expect(res.body.meta).toMatchObject({ + currentPage: 1, + itemCount: 5, + itemsPerPage: 10, + totalItems: 5, + totalPages: 1, + }) + ) + .expect((res) => expect(res.body.items[0].type).toBe(TransferType.Withdrawal)); + }); + it("returns HTTP 200 and address transfers for the specified paging configuration", () => { return request(app.getHttpServer()) .get( @@ -1207,7 +1226,7 @@ describe("AddressController (e2e)", () => { to: "0x91d0a23f34e535e44Df8Ba84c53a0945cf0eEB60", token: { l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -1218,7 +1237,7 @@ describe("AddressController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e11", type: "transfer", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, { diff --git a/packages/api/test/stats-api.e2e-spec.ts b/packages/api/test/stats-api.e2e-spec.ts index c805e8648c..aed8ed3856 100644 --- a/packages/api/test/stats-api.e2e-spec.ts +++ b/packages/api/test/stats-api.e2e-spec.ts @@ -5,11 +5,13 @@ import * as request from "supertest"; import { Repository } from "typeorm"; import { BatchDetails } from "../src/batch/batchDetails.entity"; import { BlockDetails } from "../src/block/blockDetails.entity"; -import { Token, ETH_TOKEN } from "../src/token/token.entity"; +import { Token } from "../src/token/token.entity"; import { AppModule } from "../src/app.module"; import { configureApp } from "../src/configureApp"; +import { baseTokenData } from "../src/config"; describe("Stats API (e2e)", () => { + let ETH_TOKEN; let app: INestApplication; let blockRepository: Repository; let batchRepository: Repository; @@ -19,7 +21,7 @@ describe("Stats API (e2e)", () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [AppModule], }).compile(); - + ETH_TOKEN = baseTokenData; app = moduleFixture.createNestApplication({ logger: false }); configureApp(app); await app.init(); diff --git a/packages/api/test/token-api.e2e-spec.ts b/packages/api/test/token-api.e2e-spec.ts index f3735bc8e2..7930145e7c 100644 --- a/packages/api/test/token-api.e2e-spec.ts +++ b/packages/api/test/token-api.e2e-spec.ts @@ -5,9 +5,10 @@ import * as request from "supertest"; import { Repository } from "typeorm"; import { BatchDetails } from "../src/batch/batchDetails.entity"; import { BlockDetails } from "../src/block/blockDetails.entity"; -import { Token, ETH_TOKEN } from "../src/token/token.entity"; +import { Token } from "../src/token/token.entity"; import { AppModule } from "../src/app.module"; import { configureApp } from "../src/configureApp"; +import { baseTokenData } from "../src/config"; describe("Token API (e2e)", () => { let app: INestApplication; @@ -55,16 +56,16 @@ describe("Token API (e2e)", () => { }); await tokenRepository.insert({ - l2Address: ETH_TOKEN.l2Address, - l1Address: ETH_TOKEN.l1Address, - symbol: ETH_TOKEN.symbol, - name: ETH_TOKEN.name, - decimals: ETH_TOKEN.decimals, + l2Address: baseTokenData.l2Address, + l1Address: baseTokenData.l1Address, + symbol: baseTokenData.symbol, + name: baseTokenData.name, + decimals: baseTokenData.decimals, blockNumber: 0, logIndex: 0, - usdPrice: ETH_TOKEN.usdPrice, - liquidity: ETH_TOKEN.liquidity, - iconURL: ETH_TOKEN.iconURL, + usdPrice: baseTokenData.usdPrice, + liquidity: baseTokenData.liquidity, + iconURL: baseTokenData.iconURL, }); await tokenRepository.insert({ @@ -128,21 +129,21 @@ describe("Token API (e2e)", () => { it("returns HTTP 200 and ETH token info for ETH token", () => { return request(app.getHttpServer()) - .get(`/api?module=token&action=tokeninfo&contractaddress=${ETH_TOKEN.l2Address}`) + .get(`/api?module=token&action=tokeninfo&contractaddress=${baseTokenData.l2Address}`) .expect(200) .expect((res) => expect(res.body).toStrictEqual({ message: "OK", result: [ { - contractAddress: ETH_TOKEN.l2Address, - iconURL: ETH_TOKEN.iconURL, - l1Address: ETH_TOKEN.l1Address, - liquidity: ETH_TOKEN.liquidity.toString(), - symbol: ETH_TOKEN.symbol, - tokenDecimal: ETH_TOKEN.decimals.toString(), - tokenName: ETH_TOKEN.name, - tokenPriceUSD: ETH_TOKEN.usdPrice.toString(), + contractAddress: baseTokenData.l2Address, + iconURL: baseTokenData.iconURL, + l1Address: baseTokenData.l1Address, + liquidity: baseTokenData.liquidity.toString(), + symbol: baseTokenData.symbol, + tokenDecimal: baseTokenData.decimals.toString(), + tokenName: baseTokenData.name, + tokenPriceUSD: baseTokenData.usdPrice.toString(), }, ], status: "1", diff --git a/packages/api/test/token.e2e-spec.ts b/packages/api/test/token.e2e-spec.ts index b307db3298..2afe851925 100644 --- a/packages/api/test/token.e2e-spec.ts +++ b/packages/api/test/token.e2e-spec.ts @@ -5,13 +5,15 @@ import { Repository } from "typeorm"; import { getRepositoryToken } from "@nestjs/typeorm"; import { AppModule } from "../src/app.module"; import { configureApp } from "../src/configureApp"; -import { Token, TokenType, ETH_TOKEN } from "../src/token/token.entity"; +import { Token, TokenType } from "../src/token/token.entity"; import { BlockDetails } from "../src/block/blockDetails.entity"; import { Transaction } from "../src/transaction/entities/transaction.entity"; import { Transfer, TransferType } from "../src/transfer/transfer.entity"; import { BatchDetails } from "../src/batch/batchDetails.entity"; +import { baseTokenData } from "../src/config"; describe("TokenController (e2e)", () => { + let ETH_TOKEN; let app: INestApplication; let tokenRepository: Repository; let blockRepository: Repository; @@ -23,7 +25,7 @@ describe("TokenController (e2e)", () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [AppModule], }).compile(); - + ETH_TOKEN = baseTokenData; app = moduleFixture.createNestApplication({ logger: false }); configureApp(app); @@ -232,7 +234,7 @@ describe("TokenController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", amount: "1000", type: TransferType.Refund, - tokenType: TokenType.ETH, + tokenType: TokenType.BaseToken, logIndex: transferIndex++, transactionIndex: 0, timestamp: "2022-11-21T18:16:51.000Z", @@ -314,7 +316,7 @@ describe("TokenController (e2e)", () => { .expect((res) => expect(res.body).toStrictEqual({ l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -655,7 +657,7 @@ describe("TokenController (e2e)", () => { to: "0x52312AD6f01657413b2eaE9287f6B9ADaD93D5FE", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -664,7 +666,7 @@ describe("TokenController (e2e)", () => { usdPrice: null, }, tokenAddress: "0x000000000000000000000000000000000000800A", - tokenType: "ETH", + tokenType: "BASETOKEN", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "refund", }, @@ -678,7 +680,7 @@ describe("TokenController (e2e)", () => { to: "0x52312AD6f01657413b2eaE9287f6B9ADaD93D5FE", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -687,7 +689,7 @@ describe("TokenController (e2e)", () => { usdPrice: null, }, tokenAddress: "0x000000000000000000000000000000000000800A", - tokenType: "ETH", + tokenType: "BASETOKEN", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "refund", }, @@ -701,7 +703,7 @@ describe("TokenController (e2e)", () => { to: "0x52312AD6f01657413b2eaE9287f6B9ADaD93D5FE", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -710,7 +712,7 @@ describe("TokenController (e2e)", () => { usdPrice: null, }, tokenAddress: "0x000000000000000000000000000000000000800A", - tokenType: "ETH", + tokenType: "BASETOKEN", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "refund", }, diff --git a/packages/api/test/transaction.e2e-spec.ts b/packages/api/test/transaction.e2e-spec.ts index 28f55e29f1..cc6ee721cc 100644 --- a/packages/api/test/transaction.e2e-spec.ts +++ b/packages/api/test/transaction.e2e-spec.ts @@ -10,13 +10,14 @@ import { Token, TokenType } from "../src/token/token.entity"; import { BlockDetails } from "../src/block/blockDetails.entity"; import { Transaction } from "../src/transaction/entities/transaction.entity"; import { TransactionReceipt } from "../src/transaction/entities/transactionReceipt.entity"; -import { ETH_TOKEN } from "../src/token/token.entity"; import { AddressTransaction } from "../src/transaction/entities/addressTransaction.entity"; import { Transfer, TransferType } from "../src/transfer/transfer.entity"; import { Log } from "../src/log/log.entity"; import { BatchDetails } from "../src/batch/batchDetails.entity"; +import { baseTokenData } from "../src/config"; describe("TransactionController (e2e)", () => { + let ETH_TOKEN; let app: INestApplication; let tokenRepository: Repository; let blockRepository: Repository; @@ -31,7 +32,7 @@ describe("TransactionController (e2e)", () => { const moduleFixture: TestingModule = await Test.createTestingModule({ imports: [AppModule], }).compile(); - + ETH_TOKEN = baseTokenData; app = moduleFixture.createNestApplication({ logger: false }); configureApp(app); @@ -207,7 +208,7 @@ describe("TransactionController (e2e)", () => { transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", tokenAddress: i % 2 ? "0xd754ff5e8a6f257e162f72578a4bb0493c068101" : "0x000000000000000000000000000000000000800a", - tokenType: i % 2 ? TokenType.ERC20 : TokenType.ETH, + tokenType: i % 2 ? TokenType.ERC20 : TokenType.BaseToken, amount: "2000", type, logIndex: i, @@ -1162,7 +1163,7 @@ describe("TransactionController (e2e)", () => { to: "0x52312Ad6f01657413b2eae9287F6b9Adad93d5fd", token: { l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -1173,7 +1174,7 @@ describe("TransactionController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "deposit", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, { @@ -1208,7 +1209,7 @@ describe("TransactionController (e2e)", () => { to: "0x52312Ad6f01657413b2eae9287F6b9Adad93d5fd", token: { l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -1219,7 +1220,7 @@ describe("TransactionController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "withdrawal", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, { @@ -1254,7 +1255,7 @@ describe("TransactionController (e2e)", () => { to: "0x52312Ad6f01657413b2eae9287F6b9Adad93d5fd", token: { l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -1265,7 +1266,7 @@ describe("TransactionController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "mint", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, { @@ -1300,7 +1301,7 @@ describe("TransactionController (e2e)", () => { to: "0x52312Ad6f01657413b2eae9287F6b9Adad93d5fd", token: { l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -1311,7 +1312,7 @@ describe("TransactionController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "deposit", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, { @@ -1346,7 +1347,7 @@ describe("TransactionController (e2e)", () => { to: "0x52312Ad6f01657413b2eae9287F6b9Adad93d5fd", token: { l2Address: "0x000000000000000000000000000000000000800A", - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", symbol: "ETH", name: "Ether", decimals: 18, @@ -1357,7 +1358,7 @@ describe("TransactionController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "transfer", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, ]) @@ -1414,7 +1415,7 @@ describe("TransactionController (e2e)", () => { to: "0x52312Ad6f01657413b2eae9287F6b9Adad93d5fd", token: { decimals: 18, - l1Address: "0x0000000000000000000000000000000000000000", + l1Address: "0x0000000000000000000000000000000000000001", l2Address: "0x000000000000000000000000000000000000800A", name: "Ether", symbol: "ETH", @@ -1425,7 +1426,7 @@ describe("TransactionController (e2e)", () => { tokenAddress: "0x000000000000000000000000000000000000800A", transactionHash: "0x8a008b8dbbc18035e56370abb820e736b705d68d6ac12b203603db8d9ea87e10", type: "deposit", - tokenType: "ETH", + tokenType: "BASETOKEN", isInternal: false, }, ], diff --git a/packages/app/README.md b/packages/app/README.md index 1023fc9388..692aeb3de6 100644 --- a/packages/app/README.md +++ b/packages/app/README.md @@ -46,7 +46,6 @@ const config: EnvironmentConfig = { icon: "/images/icons/zksync-arrows.svg", l2ChainId: 270, l2NetworkName: "Local", - l2WalletUrl: "https://goerli.staging-portal.zksync.dev/", maintenance: false, name: "local", published: true, @@ -60,7 +59,6 @@ const config: EnvironmentConfig = { icon: "/images/icons/zksync-arrows.svg", l2ChainId: 270, l2NetworkName: "Local Hyperchain", - l2WalletUrl: "https://goerli.staging-portal.zksync.dev/", maintenance: false, name: "local-hyperchain", published: true, @@ -106,3 +104,13 @@ npm run lint ## Production links - [Web Application](https://explorer.zksync.io) - [Storybook](https://storybook-scan-v2.zksync.dev) + + +## Verify Block Explorer UI test results in GitHub Actions +GitHub Actions test results are available in: + +- `GitHub Actions` --> `Summary` page at the very end of a page. +- Inside of each test run in the log: `Feature on Mainnet + Sepolia` --> `@search` --> `Upload test results to Allure reporter` --> `https://raw.githack.com/matter-labs/block-explorer/gh-pages/_github.run_number_/index.html` +- Directly via a link `https://raw.githack.com/matter-labs/block-explorer/gh-pages/_github.run_number_/index.html` after each PR running. The history of test runs for public view locates in `gh-pages` branch. + +In case of 404 page, make sure that the folder with its `github.run_number` exists in the `gh-pages`. If the folder exist, try again in a few minutes as `https://raw.githack.com` needs to update the data. Public report link will be available when the 'Allure Report' job will be succesfully executed. diff --git a/packages/app/mock/transactions/Execute.json b/packages/app/mock/transactions/Execute.json index 32b78b14b5..8799ac49c2 100644 --- a/packages/app/mock/transactions/Execute.json +++ b/packages/app/mock/transactions/Execute.json @@ -32,7 +32,7 @@ "l2Address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": 1 }, @@ -46,7 +46,7 @@ "l2Address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": 1 }, @@ -61,7 +61,7 @@ "l2Address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": 1 }, @@ -93,7 +93,7 @@ "l2Address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": 1 }, @@ -112,7 +112,7 @@ "l2Address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "address": "0x4732C03B2CF6eDe46500e799DE79a15Df44929eB", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": 1 }, diff --git a/packages/app/mock/transactions/ExecuteFeeOnly.json b/packages/app/mock/transactions/ExecuteFeeOnly.json index 76196f9e02..14c3b5aa21 100644 --- a/packages/app/mock/transactions/ExecuteFeeOnly.json +++ b/packages/app/mock/transactions/ExecuteFeeOnly.json @@ -22,7 +22,7 @@ "l2Address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": "1" }, @@ -37,7 +37,7 @@ "l2Address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": "1" }, @@ -69,7 +69,7 @@ "l2Address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": "1" }, @@ -88,7 +88,7 @@ "l2Address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "address": "0x4732c03b2cf6ede46500e799de79a15df44929eb", "symbol": "LINK", - "name": "ChainLink Token (goerli)", + "name": "ChainLink Token (testnet)", "decimals": 18, "usdPrice": "1" }, diff --git a/packages/app/src/App.vue b/packages/app/src/App.vue index f6e453715a..ae7892f2b5 100644 --- a/packages/app/src/App.vue +++ b/packages/app/src/App.vue @@ -2,7 +2,6 @@