Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ante handler panic in SetupContextDecorator #2520

Open
musalbas opened this issue Sep 2, 2023 · 3 comments
Open

ante handler panic in SetupContextDecorator #2520

musalbas opened this issue Sep 2, 2023 · 3 comments
Labels
bug Something isn't working WS: Maintenance 🔧 includes bugs, refactors, flakes, and tech debt etc

Comments

@musalbas
Copy link
Member

musalbas commented Sep 2, 2023

I observed this only when enabling the CAT mempool, when using 134MB blocks:

4:15PM ERR CONSENSUS FAILURE!!! err="runtime error: invalid memory address or nil pointer dereference" module=consensus stack="goroutine 3893 [running]:\nruntime/debug.Stack()\n\t/snap/go/10319/src/runtime/debug/stack.go:24 +0x5e[ngithub.com/tendermint/tendermint/consensus.(*State).receiveRoutine.func2()](http://ngithub.com/tendermint/tendermint/consensus.(*State).receiveRoutine.func2())\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:744 +0x46\npanic({0x20f9920?, 0x41294c0?})\n\t/snap/go/10319/src/runtime/panic.go:914 +0x21f[ngithub.com/cosmos/cosmos-sdk/x/auth/tx.(*wrapper).GetGas(0x38177c34074933ef?)](http://ngithub.com/cosmos/cosmos-sdk/x/auth/tx.(*wrapper).GetGas(0x38177c34074933ef?))\n\t/root/go/pkg/mod/github.com/celestiaorg/[email protected]/x/auth/tx/builder.go:133 +0xc[ngithub.com/cosmos/cosmos-sdk/x/auth/ante.SetUpContextDecorator.AnteHandle({}](http://ngithub.com/cosmos/cosmos-sdk/x/auth/ante.SetUpContextDecorator.AnteHandle(%7B%7D), {{0x2eab1a0, 0x42415c0}, {0x2ebcc90, 0xc015621e00}, {{0x0, 0x0}, {0xc001cdc957, 0x7}, 0x56a, ...}, ...}, ...)\n\t/root/go/pkg/mod/github.com/celestiaorg/[email protected]/x/auth/ante/setup.go:40 +0x2b7[ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x2eab1a0](http://ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1(%7B%7B0x2eab1a0), 0x42415c0}, {0x2ebcc90, 0xc015621e00}, {{0x0, 0x0}, {0xc001cdc957, 0x7}, 0x56a, {0x0, ...}, ...}, ...}, ...)\n\t/root/go/pkg/mod/github.com/celestiaorg/[email protected]/types/handler.go:40 +0x136[ngithub.com/celestiaorg/celestia-app/app.checkTxValidity(](http://ngithub.com/celestiaorg/celestia-app/app.checkTxValidity(_), {{0x2eab1a0, 0x42415c0}, {0x2ebcc90, 0xc015621e00}, {{0x0, 0x0}, {0xc001cdc957, 0x7}, 0x56a, ...}, ...}, ...)\n\t/root/celestia-app/app/validate_txs.go:83 +0x103[ngithub.com/celestiaorg/celestia-app/app.filterStdTxs(](http://ngithub.com/celestiaorg/celestia-app/app.filterStdTxs(_), {{0x2eab1a0, 0x42415c0}, {0x2ebcc90, 0xc015621e00}, {{0x0, 0x0}, {0xc001cdc957, 0x7}, 0x56a, ...}, ...}, ...)\n\t/root/celestia-app/app/validate_txs.go:40 +0x145[ngithub.com/celestiaorg/celestia-app/app.FilterTxs({{0x2eab1a0](http://ngithub.com/celestiaorg/celestia-app/app.FilterTxs(%7B%7B0x2eab1a0), 0x42415c0}, {0x2ebcc90, 0xc015621e00}, {{0x0, 0x0}, {0xc001cdc957, 0x7}, 0x56a, {0x0, ...}, ...}, ...}, ...)\n\t/root/celestia-app/app/validate_txs.go:28 +0xfa[ngithub.com/celestiaorg/celestia-app/app.(*App).PrepareProposal(0xc0005f3600](http://ngithub.com/celestiaorg/celestia-app/app.(*App).PrepareProposal(0xc0005f3600), {0xc015759100?, 0x19af7c5?, {0xc001cdc957?, 0xc00d307240?}})\n\t/root/celestia-app/app/prepare_proposal.go:52 +0x308[ngithub.com/tendermint/tendermint/abci/client.(*localClient).PrepareProposalSync(0xc001b2b740](http://ngithub.com/tendermint/tendermint/abci/client.(*localClient).PrepareProposalSync(0xc001b2b740), {0xc015759100?, 0x38?, {0xc001cdc957?, 0xc015b8a501?}})\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/abci/client/local_client.go:359 +0x102[ngithub.com/tendermint/tendermint/proxy.(*appConnConsensus).PrepareProposalSync(0x7f178fc4cbc8](http://ngithub.com/tendermint/tendermint/proxy.(*appConnConsensus).PrepareProposalSync(0x7f178fc4cbc8)?, {0xc015759100?, 0xffffffffffffffff?, {0xc001cdc957?, 0x80?}})\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/proxy/app_conn.go:102 +0x28[ngithub.com/tendermint/tendermint/state.(*BlockExecutor).CreateProposalBlock(_](http://ngithub.com/tendermint/tendermint/state.(*BlockExecutor).CreateProposalBlock(_), _, {{{0xb, 0x2}, {0xc001cdc950, 0x7}}, {0xc001cdc957, 0x7}, 0x1, 0x569, ...}, ...)\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/state/execution.go:118 +0x1e8[ngithub.com/tendermint/tendermint/consensus.(*State).createProposalBlock(0xc0002c6e00)](http://ngithub.com/tendermint/tendermint/consensus.(*State).createProposalBlock(0xc0002c6e00))\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:1238 +0x27b[ngithub.com/tendermint/tendermint/consensus.(*State).defaultDecideProposal(0xc0002c6e00](http://ngithub.com/tendermint/tendermint/consensus.(*State).defaultDecideProposal(0xc0002c6e00), 0x56a, 0x0)\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:1153 +0x53[ngithub.com/tendermint/tendermint/consensus.(*State).enterPropose(0xc0002c6e00](http://ngithub.com/tendermint/tendermint/consensus.(*State).enterPropose(0xc0002c6e00), 0x56a, 0x0)\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:1133 +0x83e[ngithub.com/tendermint/tendermint/consensus.(*State).enterNewRound(0xc0002c6e00](http://ngithub.com/tendermint/tendermint/consensus.(*State).enterNewRound(0xc0002c6e00), 0x56a, 0x0)\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:1052 +0xa18[ngithub.com/tendermint/tendermint/consensus.(*State).handleTimeout(0xc0002c6e00](http://ngithub.com/tendermint/tendermint/consensus.(*State).handleTimeout(0xc0002c6e00), {0xc2f41d?, 0xc0008fac40?, 0x245b2b8?, 0x0?}, {0x56a, 0x0, 0x1, {0x345a435f, 0xedc8556b9, ...}, ...})\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:927 +0x907[ngithub.com/tendermint/tendermint/consensus.(*State).receiveRoutine(0xc0002c6e00](http://ngithub.com/tendermint/tendermint/consensus.(*State).receiveRoutine(0xc0002c6e00), 0x0)\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:809 +0x650\ncreated by github.com/tendermint/tendermint/consensus.(*State).OnStart in goroutine 82\n\t/root/go/pkg/mod/github.com/musalbas/[email protected]/consensus/state.go:389 +0x10c\n"

@cmwaters
Copy link
Contributor

It looks like this stemmed from celestia-app instead of celestia-core so I will transfer the issue there. It seems like there was a panic in the antehandler. Specifically here (github.com/celestiaorg/[email protected]/x/auth/ante/setup.go:40) when calling gasTx.GetGas(). There's usually something to catch panics in the antehandler but it's declared immediately afterwards hence why this still causes a consensus panic

@cmwaters cmwaters transferred this issue from celestiaorg/celestia-core Sep 18, 2023
@cmwaters cmwaters changed the title Consensus errors when using CAT mempool with 134MB blocks ante handler panic in SetupContextDecorator Sep 18, 2023
@cmwaters
Copy link
Contributor

Im wondering if we should check for a nil tx in ValidateBasic here:

https://github.com/celestiaorg/cosmos-sdk/blob/c525ad72ab1449d31115815f5832a68bc79d4ffd/x/auth/tx/builder.go#L69

It seems to be the only place that we can panic because ValidateBasic seems to catch everything else

@musalbas
Copy link
Member Author

Branch used for 134MB blocks, for replication: musalbas@e9ed231

@rootulp rootulp added bug Something isn't working and removed needs:triage labels Sep 18, 2023
@rootulp rootulp added this to the Post-mainnet milestone Sep 18, 2023
@evan-forbes evan-forbes removed this from the Post-mainnet milestone Nov 12, 2023
@evan-forbes evan-forbes added WS: Maintenance 🔧 includes bugs, refactors, flakes, and tech debt etc needs:triage needs:investigation more knowledge is required before item can be groomed and removed external needs:investigation more knowledge is required before item can be groomed labels May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working WS: Maintenance 🔧 includes bugs, refactors, flakes, and tech debt etc
Projects
None yet
Development

No branches or pull requests

4 participants