Rustic backup to B2 dies after error instead of retrying #1326
Comments
github-actions
bot
added
the
S-triage
|
Thanks for opening this issue @ociaw In fact, yes rustic should retry here as this is classified as a temporary error - and it is even saying it is retrying. But in fact it is just stopping instead of retrying. The return code 141 indicates that rustic was terminated by a SIGPIPE signal. Can you show how you are calling the rustic binary? could it be that you pipe the output to some other program? |
|
Hmm, no piping, but I'm running it inside |
nardoor
added
C-bug
|
Hi @ociaw, in rustic 0.9.5 we improved error output - can you please retry if you see the same behavior and information using the newest version? Thanks a lot! |
|
Updated to 0.9.5, but I'm still encountering the same issue, and I don't see any additional information in the log? Same exit code, Truncated log with Log Level = Trace |
|
Ran a debug build under rust-gdb and got more information: Running Unfortunately, I'm not very familiar with GDB so I'm not sure how to get more useful information. I guess the TCP stream is closed on the other end, and rustic/opendal tries to write to it, causing the kernel to send SIGPIPE? |
|
I ended up doing the obvious thing and changing the SIGPIPE handler from Is there a reason SIG_DFL is being used instead of SIG_IGN? IMO explicitly ignoring the signal is better than letting the kernel decide what to do and getting inconsistent behavior across operating systems or distributions. |
|
@ociaw The idea about the SIGPIPE handler is to get correct error codes for failing pipe situations. I still wonder what leads to the SIGPIPE signal rustic obviously is getting in your case... |
|
Of course, we can either add an option to opt-out the sigpipe handling or maybe just handle it in very special situations (like the the |
|
FWIW, I hit the same 141 exit code issue with a REST server backend; and the issue was that my nginx reverse-proxy had not set a large enough In my case I'd say the error message could have been improved; but otherwise I wouldn't have wanted rustic to silently make it look like it worked while it didn't :) |
|
Is there a workaround for this issue? Compiling a custom binary with |
|
I have this error as well with the log: After which rustic exits with code Thank you for Rustic. |
Rustic keeps dying during my backups to B2, using opendal:b2. As a result, I have to repeatedly re-run rustic to complete the backup. During the backup, sometimes when B2 returns an error (usually
no tomes available), rustic abruptly stops with no error message and an exit code of 141. However, this doesn't always occur - other times, it correctly retries again and continues with the backup.I'm unsure why there's no error message, or what the significance of the
141exit code is. I've run rustic with the log level set to DEBUG, but that hasn't seemed to reveal anything more. Is there something more I can do to debug this better?Tested Versions: 0.8.0, 0.9.1, 0.9.3
OS: TrueNas 13.1-RELEASE-p9
Truncated rustic log with Log Level = DEBUG
My anonymized config
The text was updated successfully, but these errors were encountered: