-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bug]: LND Suddenly lots of force closes, Error while broadcasting transaction
#7959
Comments
Commenting on a part which is most likely not the reason for the Force Closes ! Analyzing the logs I think I found the improvement we could make to prevent a lot of these lines:
The anchor sweeps fail to be broadcasted because of the min-mempool fee, and this does not stop because we never run in the exhausted case because we never count up: https://github.com/lightningnetwork/lnd/blob/master/sweep/sweeper.go#L1356 and in the rebroadcaster we only check for different txid, which seems to be always the case sweeps are bundled. I think we should also count up the attempt if we fail to broadcast the transaction for specific errors. Regarding the Force-Closes, there might be a problem between the peer connection because the dangling HTLC cannot be resolved hence no stable connection can be reestablished even during 80 blocks (default cltv delta) |
Those logs look unrelated, if you look at the force closes, do they have HTLCs on them? The log areas you want to look at are |
For me the node just appeared offline and a htlc in our channel expired. |
This could've been the lock problem that we fixed in RC2. |
yeah there was an HTLC on the remote commitment ( |
Same for me. Node was unreachable for 3.5 days:
|
It looks like it, see my "CNCT" log entries added above. Could not find any relevant "HSWC" lines. |
Yeah that log line would be indicative of the bug fixed: eventually the peer sub-system would lock up, and though new TCP connections could be created, the handshake wouldn't complete in time. |
Updating to rc2 doesn't seem to have fixed my issue, still seeing force-closes. I've added more errors I could find, please find them attached to my OP. I've also noticed that I stop getting any updates on the node activity with bosbot (but I don't get a disconnect message either). When I restart LND, I get a disconnect message with bosbot. |
Are these due to normal HTLC timeouts? Those logs so far are due to transactions not being able to get into the mempool as fees are higher. If you're seeing this in a widespread manner with non anchor channels, then it may be the case that those can't confirm in time (as we can't fee bump them), so then other HTLCs start to timeout. |
If you look at the logs of |
Privately communicated LND logs to @ziggie1984 for further analysis. |
I just experienced this on RC3. Some peers force closed on restart and there were no pending htlcs |
Here are my logs for one of the channel that force closed. I can get you more logs later.
|
@niteshbalusu11 hmm the logs only show that your remote party closed the channel (force closed) would be interesting to see why your channel with this node was not able to resolve the htlcs on it? Can you share more logs before the force-close or even from your peer. |
I will share full logs soon. |
I have the same issue on 0.17.2-beta. Logs
|
Background
Nothing in particular happened. Updated to 0.17.0-rc1 shortly it came out. In the last few days, my node has seen a massive amount of force-closes.
Your environment
lnd
: LND 0.17.0-rc1uname -a
on *Nix): Linux 5.15.0-46-SMP x86_64 x86_64 x86_64 GNU/Linuxbitcoind
Steps to reproduce
Unknown.
Logs:
During the FC:
Maybe related:
More errors (update: 11.09.2023)
The text was updated successfully, but these errors were encountered: