[bug]: invalid memory address or nil pointer dereference

[hpbock]

Background

After upgrading my lnd node to v0.17.0-beta.rc1, my node fails to startup with a segmentation violation.

Your environment

• version of lnd: v0.17.0-beta.rc1
• which operating system (uname -a on *Nix): Linux 5.10.0-23-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12) x86_64 GNU/Linux
• version of btcd, bitcoind, or other backend: bitcoind
• any other relevant environment details

Steps to reproduce

lnd --accept-keysend --profile 6061 --tor.active --tor.v3 --protocol.wumbo-channels --db.prune-revocation
...
lncli unlock

Expected behaviour

I expect lnd to startup without a segmentation violation.

Actual behaviour

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xcf7a9e]

goroutine 1 [running]:
github.com/lightningnetwork/lnd/contractcourt.(*ChannelArbitrator).relaunchResolvers(0xc000598580, 0x0, 0x0?)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:739 +0x6fe
github.com/lightningnetwork/lnd/contractcourt.(*ChannelArbitrator).Start(0x146e4a0?, 0xc001406e40)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:560 +0x665
github.com/lightningnetwork/lnd/contractcourt.(*ChainArbitrator).Start(0xc002771e80?)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/contractcourt/chain_arbitrator.go:707 +0x11fe
github.com/lightningnetwork/lnd.(*server).Start.func1()
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/server.go:1987 +0x19ff
sync.(*Once).doSlow(0x3?, 0x16d9432?)
        /usr/lib/go-1.19/src/sync/once.go:74 +0xc2
sync.(*Once).Do(...)
        /usr/lib/go-1.19/src/sync/once.go:65
github.com/lightningnetwork/lnd.(*server).Start(0xc4998?)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/server.go:1862 +0x76
github.com/lightningnetwork/lnd.Main(0xc0003fa580, {{0x0?, 0xc000006340?, 0xc000066738?}}, 0xc000120fc0, {0xc0000906c0, 0xc000100720, 0xc000100780, 0xc0001007e0, {0x0}})
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/lnd.go:676 +0x3df4
main.main()
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/cmd/lnd/main.go:38 +0x205

[GeorgeTsagk]

Hello @hpbock, #7936 should fix this behavior

[hpbock]

@GeorgeTsagk thank you for your hint, but with #7936 lnd still crashes on startup with a slightly different segmentation violation:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xcf7ab9]

goroutine 1 [running]:
github.com/lightningnetwork/lnd/contractcourt.(*ChannelArbitrator).relaunchResolvers(0xc00062b080, 0x0, 0x9660fd0?)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:741 +0x6f9
github.com/lightningnetwork/lnd/contractcourt.(*ChannelArbitrator).Start(0x146e4a0?, 0xc0030556f0)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:563 +0x665
github.com/lightningnetwork/lnd/contractcourt.(*ChainArbitrator).Start(0xc00587a100?)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/contractcourt/chain_arbitrator.go:714 +0x11fe
github.com/lightningnetwork/lnd.(*server).Start.func1()
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/server.go:1987 +0x19ff
sync.(*Once).doSlow(0x3?, 0x16d9432?)
        /usr/lib/go-1.19/src/sync/once.go:74 +0xc2
sync.(*Once).Do(...)
        /usr/lib/go-1.19/src/sync/once.go:65
github.com/lightningnetwork/lnd.(*server).Start(0xc49b3?)
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/server.go:1862 +0x76
github.com/lightningnetwork/lnd.Main(0xc000586000, {{0x0?, 0xc000006340?, 0xc000066738?}}, 0xc000122e40, {0xc0000a09c0, 0xc00007e540, 0xc00007e5a0, 0xc00007e600, {0x0}})
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/lnd.go:676 +0x3df4
main.main()
        /home/lnd-mainnet/gocode/src/github.com/lightningnetwork/lnd/cmd/lnd/main.go:38 +0x205

[yyforyongyu]

@hpbock Could you try it again? Updated that PR to try a different approach.

[hpbock]

@yyforyongyu The segmentation violation is gone, but lnd does not start completely:

• lncli walletbalance runs fine and immediately returns the wallet balance after half a second
• lncli getinfo blocks since minutes without responding...

Lnd additionally complains regularly about broadcasting transactions:

...
2023-08-29 14:11:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1137
2023-08-29 14:12:41.554 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1135
2023-08-29 14:13:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1134
2023-08-29 14:14:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1133
2023-08-29 14:15:26.993 [WRN] LNWL: Unable to fetch historical channel: channel not found
2023-08-29 14:15:41.555 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1132
2023-08-29 14:16:14.008 [WRN] LNWL: Unable to fetch historical channel: channel not found
2023-08-29 14:16:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1131
2023-08-29 14:17:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1143
2023-08-29 14:18:41.554 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1143

[hpbock]

@yyforyongyu @GeorgeTsagk thank you for your support so far!

The lncli getinfo still blocks after more than 14 hours.

I now called lncli stop which returned immediately, but LND refuses to end its process.

2023-08-30 05:05:55.441 [INF] LTND: Received shutdown request.
2023-08-30 05:05:55.441 [INF] LTND: Shutting down...
2023-08-30 05:05:55.441 [INF] LTND: Gracefully shutting down.
2023-08-30 05:06:13.476 [WRN] LNWL: Unable to fetch historical channel: channel not found
2023-08-30 05:06:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1064
2023-08-30 05:07:41.553 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1063
2023-08-30 05:08:41.554 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 165 < 1062

Should I create another issue for these new problems?

[yyforyongyu]

@hpbock thank you for reporting this issue! We are aware of the getinfo issue and working on it.