[evm] bug where successful transactions hang even when they succeed

[Unique-Divine]

Problem Evidence 1

Nibiru:

If the txs are hanging, they may be failing silently. Could you share the settings you're using to broadcast the transaction, so we can help debug?

Via Labs:

https://evm-rpc.archive.devnet-3.nibiru.fi
Chain ID: 7222

We are on this one, we also tried the non-archive one
It does seem like they are hanging with no specific error message
We are sending the tx in a standard way, no special params that I know of are supplied

It's a regular transaction to our message gateway contract, all I have is the computed hash, I don't have a standalone script to share no sorry

Nibiru:

Maybe taking a step back, have you completed any successful transactions on the chain?

Via Labs:

Simple transfers or deploying other contracts, for example

There have been some config and transfers from quite a while ago when we initially deployed

Is there anything you can look up in your network with that hash? I don't believe we can do that from the explorer

In general the system then just works, as we have it running on over 250 testnetd and mainnets so that's why I'm a bit lost what to kick on our side

Note on Forge/Foundry

• We recently added support for this in
  feat(evm-forge): foundry support and template for Nibiru EVM develoment #2084

Problem Evidence 2

"Mind telling me the commands you're using? Would still like to replicate the
behavior and improve the UX for others"

For example, when interacting with a Solidity contract to execute a tx:

cast send --rpc-url=http://127.0.0.1:8545 0xF8Da4a4A57e4aFBdeA4c541DCa626a47Ed874729 \
  "test(uint256)" 1337 \
  --mnemonic="guard cream sadness conduct invite crumble clock pudding hole grit liar hotel maid produce squeeze return argue turtle know drive eight casino maze host" \
  --nonce=1 --gas-price=10 \
  --value="1ether"

or when deploying a contract via Forge,

forge create Nibiru --rpc-url=http://127.0.0.1:8545 \
  --mnemonic="guard cream sadness conduct invite crumble clock pudding hole grit liar hotel maid produce squeeze return argue turtle know drive eight casino maze host" \
  --nonce=2 --gas-price=10

In both cases, the cli command did not receive a response from the node and it seems like the command was stuck. But the node successfully processed the tx.

Also,

cast receipt --rpc-url=http://127.0.0.1:8545 0x7d2de1e5d8fd4b6bad52b8778817a1ec8c8617cccc5f9f4840524888aef3c6a6

to retrieve the TX receipt returned the following error:

Deserialization Error: invalid length 0, expected a (both 0x-prefixed or not) hex string or byte array containing 32 bytes at line 1 column 25. Response: {"type":"0x2","root":"0x","status":"0x1","cumulativeGasUsed":"0xda83","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logs":[],"transactionHash":"0x7d2de1e5d8fd4b6bad52b8778817a1ec8c8617cccc5f9f4840524888aef3c6a6","contractAddress":"0x0000000000000000000000000000000000000000","gasUsed":"0xda83","blockHash":"0xcab9f968381b7c1cf4bad0340cbeb3fff29fa0242aa92c7465ad822d578c60d0","blockNumber":"0x295","transactionIndex":"0x0"}

Haven't verified the root cause though

[Unique-Divine]

• Related fix(evm): removed deprecated root key from eth_getTransactionReceipt #2108