Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build #1721

Merged
merged 13 commits into from
Dec 27, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ jobs:
integration/.build/faucet/*
integration/.build/obscuroscan/*
integration/.build/tengateway/*
integration/.build/smartcontracts/*
retention-days: 1


Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ const func: DeployFunction = async function (hre: HardhatRuntimeEnvironment) {
return l2Network.deployments.execute("CrossChainMessenger", {
from: l2Accounts.deployer,
log: true,
gasLimit: 2_500_000
gasLimit: 5_000_000
}, "relayMessage", msg);
};

Expand Down
4 changes: 2 additions & 2 deletions go/config/enclave_cli_flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,14 @@ var EnclaveFlags = map[string]*flag.TenFlag{
MaxRollupSizeFlag: flag.NewUint64Flag(MaxRollupSizeFlag, 1024*64, "The maximum size a rollup is allowed to reach"),
L2BaseFeeFlag: flag.NewUint64Flag(L2BaseFeeFlag, 1, ""),
L2CoinbaseFlag: flag.NewStringFlag(L2CoinbaseFlag, "0xd6C9230053f45F873Cb66D8A02439380a37A4fbF", ""),
GasBatchExecutionLimit: flag.NewUint64Flag(GasBatchExecutionLimit, 3_000_000, "Max gas that can be executed in a single batch"),
GasBatchExecutionLimit: flag.NewUint64Flag(GasBatchExecutionLimit, 30_000_000, "Max gas that can be executed in a single batch"),
ObscuroGenesisFlag: flag.NewStringFlag(ObscuroGenesisFlag, "", "The json string with the obscuro genesis"),
L1ChainIDFlag: flag.NewInt64Flag(L1ChainIDFlag, 1337, "An integer representing the unique chain id of the Ethereum chain used as an L1 (default 1337)"),
ObscuroChainIDFlag: flag.NewInt64Flag(ObscuroChainIDFlag, 443, "An integer representing the unique chain id of the Obscuro chain (default 443)"),
UseInMemoryDBFlag: flag.NewBoolFlag(UseInMemoryDBFlag, true, "Whether the enclave will use an in-memory DB rather than persist data"),
ProfilerEnabledFlag: flag.NewBoolFlag(ProfilerEnabledFlag, false, "Runs a profiler instance (Defaults to false)"),
DebugNamespaceEnabledFlag: flag.NewBoolFlag(DebugNamespaceEnabledFlag, false, "Whether the debug namespace is enabled"),
GasLocalExecutionCapFlag: flag.NewUint64Flag(GasLocalExecutionCapFlag, 3_000_000, "Max gas usage when executing local transactions"),
GasLocalExecutionCapFlag: flag.NewUint64Flag(GasLocalExecutionCapFlag, 40_000_000, "Max gas usage when executing local transactions"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it now set to 40 M, while GasBatchExecutionLimit is 30M?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might have a bug somewhere, but it seems like if the GasLocalExecutionCapFlag <= GasBatchExecutionLimit we get more flaky errors, so I ended up bumping the local execution max gas to more than the batch size.

}

// enclaveRestrictedFlags are the flags that the enclave can receive ONLY over the Ego signed enclave.json
Expand Down
4 changes: 2 additions & 2 deletions go/enclave/crosschain/message_bus_manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ func (m *MessageBusManager) GenerateMessageBusDeployTx() (*common.L2Tx, error) {
tx := &types.LegacyTx{
Nonce: 0, // The first transaction of the owner identity should always be deploying the contract
Value: gethcommon.Big0,
Gas: 2_000_000, // It's quite the expensive contract.
Gas: 5_000_000, // It's quite the expensive contract.
GasPrice: gethcommon.Big0, // Synthetic transactions are on the house. Or the house.
Data: gethcommon.FromHex(MessageBus.MessageBusMetaData.Bin),
To: nil, // Geth requires nil instead of gethcommon.Address{} which equates to zero address in order to return receipt.
Expand Down Expand Up @@ -219,7 +219,7 @@ func (m *MessageBusManager) CreateSyntheticTransactions(messages common.CrossCha
tx := &types.LegacyTx{
Nonce: startingNonce + uint64(idx),
Value: gethcommon.Big0,
Gas: 2_000_000,
Gas: 5_000_000,
GasPrice: gethcommon.Big0, // Synthetic transactions are on the house. Or the house.
Data: data,
To: m.messageBusAddress,
Expand Down
19 changes: 11 additions & 8 deletions go/enclave/enclave.go
Original file line number Diff line number Diff line change
Expand Up @@ -685,15 +685,18 @@ func (e *enclaveImpl) GetTransactionCount(encryptedParams common.EncryptedParams
}

var nonce uint64
l2Head, err := e.storage.FetchBatchBySeqNo(e.registry.HeadBatchSeq().Uint64())
if err == nil {
// todo - we should return an error when head state is not available, but for current test situations with race
// conditions we allow it to return zero while head state is uninitialized
s, err := e.storage.CreateStateDB(l2Head.Hash())
if err != nil {
return nil, responses.ToInternalError(err)
headBatch := e.registry.HeadBatchSeq()
if headBatch != nil {
l2Head, err := e.storage.FetchBatchBySeqNo(headBatch.Uint64())
if err == nil {
// todo - we should return an error when head state is not available, but for current test situations with race
// conditions we allow it to return zero while head state is uninitialized
s, err := e.storage.CreateStateDB(l2Head.Hash())
if err != nil {
return nil, responses.ToInternalError(err)
}
nonce = s.GetNonce(address)
}
nonce = s.GetNonce(address)
}

encoded := hexutil.EncodeUint64(nonce)
Expand Down
1 change: 1 addition & 0 deletions go/enclave/nodetype/sequencer.go
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ func (s *sequencer) createGenesisBatch(block *common.L1Block) error {
return err
}

time.Sleep(time.Second)
otherview marked this conversation as resolved.
Show resolved Hide resolved
// produce batch #2 which has the message bus and any other system contracts
cb, err := s.produceBatch(
big.NewInt(0).Add(batch.Header.SequencerOrderNo, big.NewInt(1)),
Expand Down
4 changes: 2 additions & 2 deletions integration/common/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ func DefaultEnclaveConfig() *config.EnclaveConfig {
MaxRollupSize: 1024 * 64,
GasPaymentAddress: gethcommon.HexToAddress("0xd6C9230053f45F873Cb66D8A02439380a37A4fbF"),
BaseFee: new(big.Int).SetUint64(1),
GasLocalExecutionCapFlag: 3_000_000,
GasBatchExecutionLimit: 3_000_000,
GasBatchExecutionLimit: 30_000_000,
GasLocalExecutionCapFlag: 40_000_000,
}
}
2 changes: 1 addition & 1 deletion integration/common/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ func PrefundWallets(ctx context.Context, faucetWallet wallet.Wallet, faucetClien
tx := &types.LegacyTx{
Nonce: startingNonce + uint64(idx),
Value: alloc,
Gas: uint64(1_000_000),
Gas: uint64(100_000),
GasPrice: gethcommon.Big1,
To: &destAddr,
}
Expand Down
30 changes: 15 additions & 15 deletions integration/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@ package integration

// Tracks the start ports handed out to different tests, in a bid to minimise conflicts.
const (
StartPortEth2NetworkTests = 31000
StartPortNodeRunnerTest = 32000
StartPortSimulationGethInMem = 34000
StartPortSimulationInMem = 35000
StartPortSimulationFullNetwork = 37000
StartPortSmartContractTests = 38000
StartPortContractDeployerTest = 39000
StartPortWalletExtensionUnitTest = 40000
StartPortFaucetUnitTest = 41000
StartPortFaucetHTTPUnitTest = 42000
StartPortTenscanUnitTest = 43000
StartPortTenGatewayUnitTest = 44000
StartPortEth2NetworkTests = 10000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason we need to change ports? Were there conflicts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right. There are still conflicts in ports, although less now. I've also added a log to tell us which process is occupying the conflicting port.

StartPortNodeRunnerTest = 14000
StartPortSimulationGethInMem = 18000
StartPortSimulationInMem = 22000
StartPortSimulationFullNetwork = 26000
StartPortSmartContractTests = 30000
StartPortContractDeployerTest = 34000
StartPortWalletExtensionUnitTest = 38000
StartPortFaucetUnitTest = 42000
StartPortFaucetHTTPUnitTest = 48000
StartPortTenscanUnitTest = 52000
StartPortTenGatewayUnitTest = 56000

DefaultGethWSPortOffset = 100
DefaultGethAUTHPortOffset = 200
Expand All @@ -24,9 +24,9 @@ const (
DefaultEnclaveOffset = 700 // The default offset between a Geth nodes port and the enclave ports. Used in Socket Simulations.
DefaultHostRPCHTTPOffset = 800 // The default offset for the host's RPC HTTP port
DefaultHostRPCWSOffset = 900 // The default offset for the host's RPC websocket port
DefaultTenscanHTTPPortOffset = 910
DefaultTenGatewayHTTPPortOffset = 930
DefaultTenGatewayWSPortOffset = 940
DefaultTenscanHTTPPortOffset = 1000
DefaultTenGatewayHTTPPortOffset = 1001
DefaultTenGatewayWSPortOffset = 1002
)

const (
Expand Down
36 changes: 36 additions & 0 deletions integration/eth2network/eth2_network.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ import (
"github.com/ethereum/go-ethereum/ethclient"
"github.com/ten-protocol/go-ten/integration/datagenerator"
"golang.org/x/sync/errgroup"

gethcommon "github.com/ethereum/go-ethereum/common"
)

const (
Expand Down Expand Up @@ -206,6 +208,10 @@ func (n *Impl) Start() error {
startTime := time.Now()
var eg errgroup.Group

if err := n.ensureNoDuplicatedNetwork(); err != nil {
return err
}

// initialize the genesis data on the nodes
for _, nodeDataDir := range n.dataDirs {
dataDir := nodeDataDir
Expand Down Expand Up @@ -506,6 +512,11 @@ func (n *Impl) waitForMergeEvent(startTime time.Time) error {
}

fmt.Printf("Reached the merge block after %s\n", time.Since(startTime))

if err = n.prefundedBalancesActive(dial); err != nil {
fmt.Printf("Error prefunding accounts %s\n", err.Error())
return err
}
return nil
}

Expand Down Expand Up @@ -589,6 +600,31 @@ func (n *Impl) gethImportEnodes(enodes []string) error {
return nil
}

func (n *Impl) prefundedBalancesActive(client *ethclient.Client) error {
for _, addr := range n.preFundedMinerAddrs {
balance, err := client.BalanceAt(context.Background(), gethcommon.HexToAddress(addr), nil)
if err != nil {
return fmt.Errorf("unable to check balance for account %s - %w", addr, err)
}
if balance.Cmp(gethcommon.Big0) == 0 {
return fmt.Errorf("unexpected %s balance for account %s", balance.String(), addr)
}
fmt.Printf("Account %s prefunded with %s\n", addr, balance.String())
}

return nil
}

func (n *Impl) ensureNoDuplicatedNetwork() error {
for nodeIdx, port := range n.gethWSPorts {
_, err := ethclient.Dial(fmt.Sprintf("ws://127.0.0.1:%d", port))
if err == nil {
return fmt.Errorf("unexpected geth node %d is active before the network is started", nodeIdx)
}
}
return nil
}
otherview marked this conversation as resolved.
Show resolved Hide resolved

func min(a, b int) int {
if a < b {
return a
Expand Down
40 changes: 40 additions & 0 deletions integration/simulation/network/socket.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
package network

import (
"bufio"
"fmt"
"os/exec"
"regexp"
"strings"
"time"

"github.com/ten-protocol/go-ten/integration/noderunner"
Expand Down Expand Up @@ -112,6 +116,10 @@ func (n *networkOfSocketNodes) Create(simParams *params.SimParams, _ *stats.Stat
// start the nodes
err = nodes[i].Start()
if err != nil {
errCheck := checkProcessPort(err.Error())
if errCheck != nil {
testlog.Logger().Warn("no port found on error", log.ErrKey, err)
}
testlog.Logger().Crit("unable to start obscuro node ", log.ErrKey, err)
}
}
Expand Down Expand Up @@ -181,3 +189,35 @@ func (n *networkOfSocketNodes) createConnections(simParams *params.SimParams) er
}
return nil
}

// getProcessesUsingPort returns a slice of process details using the specified port.
func checkProcessPort(errPort string) error {
re := regexp.MustCompile(`:(\d+):`)
matches := re.FindStringSubmatch(errPort)

if len(matches) < 2 {
return fmt.Errorf("no port found in string")
}

port := matches[1]

cmd := exec.Command("lsof", "-i", fmt.Sprintf(":%s", port)) //nolint:gosec

output, err := cmd.Output()
if err != nil {
return err
}

var processes []string
scanner := bufio.NewScanner(strings.NewReader(string(output)))
for scanner.Scan() {
line := scanner.Text()
if strings.Contains(line, "LISTEN") || strings.Contains(line, "ESTABLISHED") {
processes = append(processes, line)
}
}

fmt.Printf("Found processes still opened on port %s - %+v\n", port, processes)

return nil
}
2 changes: 1 addition & 1 deletion integration/simulation/simulation.go
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ func (s *Simulation) deployObscuroERC20s() {

deployContractTx := types.DynamicFeeTx{
Nonce: NextNonce(s.ctx, s.RPCHandles, owner),
Gas: 2_000_000,
Gas: 5_000_000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for increasing gas here, while decreasing it in func PrefundWallets?
Were previous estimates way off and we are adjusting to make them more optimised?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just making the gas limit equal everywhere pretty much.
Also, I'm not sure why we're using the DynamicFee tx in that particular tx and didn't want to over tweak it.

GasFeeCap: gethcommon.Big1, // This field is used to derive the gas price for dynamic fee transactions.
Data: contractBytes,
GasTipCap: gethcommon.Big1,
Expand Down
2 changes: 1 addition & 1 deletion integration/smartcontract/smartcontracts_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ var testLogs = "../.build/noderunner/"
func init() { //nolint:gochecknoinits
testlog.Setup(&testlog.Cfg{
LogDir: testLogs,
TestType: "noderunner",
TestType: "smartcontracts",
TestSubtype: "test",
LogLevel: log.LvlInfo,
})
Expand Down
2 changes: 1 addition & 1 deletion tools/hardhatdeployer/contract_deployer.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ func (cd *contractDeployer) run() (string, error) {
deployContractTx := types.LegacyTx{
Nonce: cd.wallet.GetNonceAndIncrement(),
GasPrice: big.NewInt(1),
Gas: uint64(2_000_000),
Gas: uint64(5_000_000),
Data: cd.contractCode,
}

Expand Down
Loading