Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build #1721

Merged
merged 13 commits into from
Dec 27, 2023
Merged

Fix build #1721

merged 13 commits into from
Dec 27, 2023

Conversation

otherview
Copy link
Contributor

@otherview otherview commented Dec 22, 2023

Why this change is needed

There are a bunch of latent issues that seem to creep up when the gas limits are added.
I have a few guesses on what might be happening, mainly relating with contract sizes being large, and blocks getting filled faster than the testing is used to. However I think this needs to be more carefully inspected.

This PR addresses the following:

  • Re-adds all reverted PRs:
    Gaslimit + gascap flags #1682
    Pedro/debug hh deployer #1710
    Ziga/fix gateway spammy logs #1715

  • Adds missing log folders ( faucet, obscuroscan, tengateway, smartcontracts )

  • Fixes gas limits on calls on both go code and HardHat

  • Bump gas values to 30M ( Re-opened the ticket to reduce max batch size to 2.5M )

  • Bump ports - now have a 2k distance between them

  • Adds 1 sec delay between Genesis and Batch Protects against nil pointer in findRoundWinner. #2 ( producing 2 batches so close together was leading the synthetic contract deployment to be all screwy leading to complaints on the nonce and then to an unreliable batch state)

  • Added ensureNoDuplicatedNetwork for geth to ensure no lingering network is running when it shouldnt

  • Added prefundedBalancesActive making sure the prefunded balances are ok at the end of the eth2network start cycle

What changes were made as part of this PR

Please provide a high level list of the changes made

PR checks pre-merging

Please indicate below by ticking the checkbox that you have read and performed the required
PR checks

  • PR checks reviewed and performed

Copy link

coderabbitai bot commented Dec 22, 2023

Warning

Rate Limit Exceeded

@otherview has exceeded the limit for the number of files or commits that can be reviewed per hour. Please wait 1 minutes and 12 seconds before requesting another review.

How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.
Please see our FAQ for further information.

Commits Files that changed from the base of the PR and between 6f9c00e and 7bad38d.

Walkthrough

The updates reflect significant adjustments to gas limits for batch executions, local transactions, and smart contract deployments, suggesting optimizations for higher throughput or more complex operations. Port assignments for tests and services have been methodically increased, likely to avoid conflicts. A specific delay has been introduced in the sequencer logic, and additional checks for head state availability have been implemented. The integration and deployment scripts have been fine-tuned, with changes to test configurations and build retention policies.

Changes

File Summary
go/config/enclave_cli_flags.go
integration/common/constants.go
Increased GasBatchExecutionLimit from 3,000,000 to 30,000,000 and GasLocalExecutionCapFlag from 3,000,000 to 40,000,000.
integration/common/utils.go Modified Gas value in PrefundWallets function from 1,000,000 to 100,000.
integration/constants.go Incremented start port values by 1000 for various tests and services, updating the corresponding constants.
integration/simulation/simulation.go
tools/hardhatdeployer/contract_deployer.go
contracts/deployment_scripts/testnet/layer2/001_whitelist_tokens.ts
Changed Gas limit from 2,000,000 (or 2,500,000) to 5,000,000 for contract deployments and specific function calls.
go/enclave/nodetype/sequencer.go Introduced a 1-second sleep delay before producing batch #2.
.github/workflows/build-pr.yml Added integration/.build/smartcontracts/* to paths and set retention-days to 1.
go/enclave/enclave.go Added a conditional check for head state availability in GetTransactionCount method.
integration/smartcontract/smartcontracts_test.go Changed TestType value from "noderunner" to "smartcontracts" within the init function.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • You can directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
  • You can tag CodeRabbit on specific lines of code or entire files in the PR by tagging @coderabbitai in a comment. Examples:
    • @coderabbitai generate unit tests for this file
    • @coderabbitai modularize this function
  • You can tag @coderabbitai in a PR comment and ask questions about the PR and the codebase. Examples:
    • @coderabbitai gather interesting statistics about this repository and render them in a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai generate unit tests for the src/utils.ts file.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between c990832 and 60a45ee.
Files selected for processing (6)
  • go/config/enclave_cli_flags.go (1 hunks)
  • go/enclave/crosschain/message_bus_manager.go (2 hunks)
  • integration/common/constants.go (1 hunks)
  • integration/constants.go (2 hunks)
  • integration/simulation/simulation.go (1 hunks)
  • tools/hardhatdeployer/contract_deployer.go (1 hunks)
Additional comments: 9
integration/constants.go (2)
  • 6-16: The changes to the port numbers are consistent and appear to be correctly incremented by 1000 to avoid conflicts or align with new strategies.

  • 27-29: The updates to the port offset constants are consistent and seem to standardize the offset to 1000, which may simplify configuration or align with other system changes.

integration/common/constants.go (1)
  • 86-87: The changes to the gas limit constants from 3,000,000 to 5,000,000 are consistent with the PR's objectives to increase computational resources.
go/config/enclave_cli_flags.go (2)
  • 58-58: The increase of GasBatchExecutionLimit from 3,000,000 to 5,000,000 aligns with the PR's objectives to handle more resource-intensive operations.

  • 65-65: The increase of GasLocalExecutionCapFlag from 3,000,000 to 5,000,000 aligns with the PR's objectives to handle more resource-intensive operations.

tools/hardhatdeployer/contract_deployer.go (1)
  • 95-95: The update of the Gas field within the deployContractTx struct from 2,000,000 to 4,000,000 is consistent with the PR's objectives to accommodate more complex or resource-intensive contracts.
go/enclave/crosschain/message_bus_manager.go (2)
  • 89-89: The increase of the gas limit in GenerateMessageBusDeployTx from 2,000,000 to 4,000,000 aligns with the PR's objectives to handle more resource-intensive operations.

  • 222-222: The increase of the gas limit in CreateSyntheticTransactions from 2,000,000 to 4,900,000 aligns with the PR's objectives to handle more resource-intensive operations.

integration/simulation/simulation.go (1)
  • 230-230: The update of the Gas field within the deployContractTx from 2,000,000 to 4,000,000 is consistent with the PR's objectives to accommodate more complex or resource-intensive contracts.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 60a45ee and e79e913.
Files selected for processing (2)
  • integration/simulation/simulation.go (1 hunks)
  • tools/hardhatdeployer/contract_deployer.go (1 hunks)
Files skipped from review as they are similar to previous changes (2)
  • integration/simulation/simulation.go
  • tools/hardhatdeployer/contract_deployer.go

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between e79e913 and 3792c75.
Files selected for processing (1)
  • integration/constants.go (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • integration/constants.go

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 3792c75 and 7aaafdd.
Files selected for processing (5)
  • go/config/enclave_cli_flags.go (1 hunks)
  • go/enclave/crosschain/message_bus_manager.go (2 hunks)
  • integration/common/constants.go (1 hunks)
  • integration/simulation/simulation.go (1 hunks)
  • tools/hardhatdeployer/contract_deployer.go (1 hunks)
Files skipped from review as they are similar to previous changes (5)
  • go/config/enclave_cli_flags.go
  • go/enclave/crosschain/message_bus_manager.go
  • integration/common/constants.go
  • integration/simulation/simulation.go
  • tools/hardhatdeployer/contract_deployer.go

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 7aaafdd and fdb8c05.
Files selected for processing (6)
  • contracts/deployment_scripts/testnet/layer2/001_whitelist_tokens.ts (1 hunks)
  • go/config/enclave_cli_flags.go (1 hunks)
  • go/enclave/crosschain/message_bus_manager.go (2 hunks)
  • integration/common/constants.go (1 hunks)
  • integration/simulation/simulation.go (1 hunks)
  • tools/hardhatdeployer/contract_deployer.go (1 hunks)
Files skipped from review due to trivial changes (1)
  • contracts/deployment_scripts/testnet/layer2/001_whitelist_tokens.ts
Files skipped from review as they are similar to previous changes (3)
  • go/enclave/crosschain/message_bus_manager.go
  • integration/simulation/simulation.go
  • tools/hardhatdeployer/contract_deployer.go
Additional comments: 3
integration/common/constants.go (1)
  • 86-87: The increase in gas limits from 3_000_000 to 10_000_000 for GasLocalExecutionCapFlag and to 12_000_000 for GasBatchExecutionLimit should be validated to ensure that these new limits are in line with the system's requirements and do not introduce any unexpected behavior or resource constraints.
go/config/enclave_cli_flags.go (2)
  • 58-58: The update to GasBatchExecutionLimit flag's default value from 3_000_000 to 10_000_000 is a significant change. Ensure that this new default is consistent with the system's expected behavior and that it has been thoroughly tested to handle the increased gas limit without issues.

  • 65-65: The update to GasLocalExecutionCapFlag flag's default value from 3_000_000 to 12_000_000 is a significant change. Ensure that this new default is consistent with the system's expected behavior and that it has been thoroughly tested to handle the increased gas limit without issues.

integration/common/constants.go Outdated Show resolved Hide resolved
go/config/enclave_cli_flags.go Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between fdb8c05 and 57dcce5.
Files selected for processing (4)
  • contracts/deployment_scripts/testnet/layer2/001_whitelist_tokens.ts (1 hunks)
  • go/config/enclave_cli_flags.go (1 hunks)
  • integration/common/constants.go (1 hunks)
  • integration/eth2network/eth2_network.go (4 hunks)
Files skipped from review as they are similar to previous changes (1)
  • contracts/deployment_scripts/testnet/layer2/001_whitelist_tokens.ts
Additional comments: 5
integration/common/constants.go (1)
  • 86-87: The increase in gas limits for GasLocalExecutionCapFlag and GasBatchExecutionLimit is substantial. Ensure that these new limits have been tested to not negatively impact the system's performance or cost.
go/config/enclave_cli_flags.go (1)
  • 58-65: The increase in gas limits for GasBatchExecutionLimit and GasLocalExecutionCapFlag is substantial. Ensure that these new limits have been tested to not negatively impact the system's performance or cost.
integration/eth2network/eth2_network.go (3)
  • 22-22: The addition of the gethcommon import is noted. Ensure that this package is used effectively in the new methods introduced.

  • 211-213: The integration of ensureNoDuplicatedNetwork in the Start method is correct and ensures that the network is not started if there are duplicated nodes running. This is a good practice for network integrity.

  • 516-518: The call to prefundedBalancesActive is correctly placed after the merge event to ensure that the prefunded accounts are active. This is a good addition for validating the network's initial state.

integration/eth2network/eth2_network.go Show resolved Hide resolved
integration/eth2network/eth2_network.go Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 57dcce5 and a58c598.
Files selected for processing (1)
  • go/enclave/nodetype/sequencer.go (1 hunks)
Files skipped from review due to trivial changes (1)
  • go/enclave/nodetype/sequencer.go

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between a58c598 and d12a540.
Files selected for processing (2)
  • integration/common/constants.go (1 hunks)
  • integration/common/utils.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
  • integration/common/constants.go
Additional comments: 1
integration/common/utils.go (1)
  • 102-102: The gas value for transactions in PrefundWallets has been reduced from 1,000,000 to 100,000. Ensure this change does not lead to failed transactions due to out-of-gas errors, especially if the transactions involve complex operations that require more gas.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between d12a540 and 30f5fd8.
Files selected for processing (2)
  • go/enclave/nodetype/sequencer.go (1 hunks)
  • integration/eth2network/eth2_network.go (4 hunks)
Files skipped from review as they are similar to previous changes (2)
  • go/enclave/nodetype/sequencer.go
  • integration/eth2network/eth2_network.go

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 1

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 30f5fd8 and 51a8fdc.
Files selected for processing (3)
  • .github/workflows/build-pr.yml (1 hunks)
  • go/enclave/enclave.go (1 hunks)
  • integration/smartcontract/smartcontracts_test.go (1 hunks)
Files skipped from review due to trivial changes (1)
  • .github/workflows/build-pr.yml
Additional comments: 3
integration/smartcontract/smartcontracts_test.go (2)
  • 41-41: The change of TestType from "noderunner" to "smartcontracts" should be consistent with the test suite's configuration. Confirm that this change aligns with the intended test categorization and that any dependent configurations or documentation have been updated accordingly.

  • 38-44: The modification in the init function appears to be straightforward and isolated to the test configuration. If the change of TestType has been verified and is correct, the rest of the hunk looks good.

go/enclave/enclave.go (1)
  • 688-693: The changes in the GetTransactionCount method introduce a conditional check for an uninitialized head state, returning zero in such a case. While this may be intended for test scenarios with race conditions, it's important to ensure that this behavior is acceptable in production or if it should be guarded by a feature flag or environment check to prevent unintended side effects in a production environment.

go/enclave/enclave.go Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 30f5fd8 and 3bde2ac.
Files selected for processing (3)
  • .github/workflows/build-pr.yml (1 hunks)
  • go/enclave/enclave.go (1 hunks)
  • integration/smartcontract/smartcontracts_test.go (1 hunks)
Files skipped from review as they are similar to previous changes (3)
  • .github/workflows/build-pr.yml
  • go/enclave/enclave.go
  • integration/smartcontract/smartcontracts_test.go

@otherview
Copy link
Contributor Author

@CodeRabbit pause

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 3bde2ac and 0d39bae.
Files selected for processing (1)
  • integration/simulation/network/socket.go (3 hunks)
Additional comments: 3
integration/simulation/network/socket.go (3)
  • 4-8: The addition of imports "bufio," "os/exec," "regexp," and "strings" is noted. Ensure these are used within the file and are not extraneous to avoid unnecessary dependencies.

  • 119-122: The error handling in node startup now includes a call to checkProcessPort. Ensure that this function is robust and correctly interprets errors related to port issues.

  • 194-223: The new function checkProcessPort uses system commands to check for port usage. Verify that this does not introduce security risks, such as command injection, and that it performs adequately without unnecessary resource usage.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 3bde2ac and 6f9c00e.
Files selected for processing (1)
  • integration/simulation/network/socket.go (3 hunks)
Files skipped from review as they are similar to previous changes (1)
  • integration/simulation/network/socket.go

@otherview
Copy link
Contributor Author

Stabilized Unit tests
image

The red failure is a port re-use.

@otherview
Copy link
Contributor Author

Copy link
Contributor

@zkokelj zkokelj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Just minor comments / questions about the logic behind the changes.

ObscuroGenesisFlag: flag.NewStringFlag(ObscuroGenesisFlag, "", "The json string with the obscuro genesis"),
L1ChainIDFlag: flag.NewInt64Flag(L1ChainIDFlag, 1337, "An integer representing the unique chain id of the Ethereum chain used as an L1 (default 1337)"),
ObscuroChainIDFlag: flag.NewInt64Flag(ObscuroChainIDFlag, 443, "An integer representing the unique chain id of the Obscuro chain (default 443)"),
UseInMemoryDBFlag: flag.NewBoolFlag(UseInMemoryDBFlag, true, "Whether the enclave will use an in-memory DB rather than persist data"),
ProfilerEnabledFlag: flag.NewBoolFlag(ProfilerEnabledFlag, false, "Runs a profiler instance (Defaults to false)"),
DebugNamespaceEnabledFlag: flag.NewBoolFlag(DebugNamespaceEnabledFlag, false, "Whether the debug namespace is enabled"),
GasLocalExecutionCapFlag: flag.NewUint64Flag(GasLocalExecutionCapFlag, 3_000_000, "Max gas usage when executing local transactions"),
GasLocalExecutionCapFlag: flag.NewUint64Flag(GasLocalExecutionCapFlag, 40_000_000, "Max gas usage when executing local transactions"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it now set to 40 M, while GasBatchExecutionLimit is 30M?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might have a bug somewhere, but it seems like if the GasLocalExecutionCapFlag <= GasBatchExecutionLimit we get more flaky errors, so I ended up bumping the local execution max gas to more than the batch size.

go/enclave/nodetype/sequencer.go Show resolved Hide resolved
StartPortFaucetHTTPUnitTest = 42000
StartPortTenscanUnitTest = 43000
StartPortTenGatewayUnitTest = 44000
StartPortEth2NetworkTests = 10000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason we need to change ports? Were there conflicts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right. There are still conflicts in ports, although less now. I've also added a log to tell us which process is occupying the conflicting port.

@@ -227,7 +227,7 @@ func (s *Simulation) deployObscuroERC20s() {

deployContractTx := types.DynamicFeeTx{
Nonce: NextNonce(s.ctx, s.RPCHandles, owner),
Gas: 2_000_000,
Gas: 5_000_000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for increasing gas here, while decreasing it in func PrefundWallets?
Were previous estimates way off and we are adjusting to make them more optimised?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just making the gas limit equal everywhere pretty much.
Also, I'm not sure why we're using the DynamicFee tx in that particular tx and didn't want to over tweak it.

@otherview otherview merged commit 520ea1e into main Dec 27, 2023
3 checks passed
@otherview otherview deleted the pedro/fix_build branch December 27, 2023 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants