Bug: referenced path cannot exist #7
Comments
|
I'm currently testing the sh version |
|
Hey @Mek101, |
|
Hitting the same problem. Here is some debug log with the problem node: |
|
Hi @vbargsten, |
|
Woops close by miss clicking |
|
@vbargsten could you produce a btrfs stream file to help me produce a test case ? As far as I understand (I quickly read the DEBUG you kindly provided) : |
|
Hi @vbargsten and @Mek101 , If, by any chance, you could checkout again the project (beware, I had to rewrite the git history, for a personal reason @Mek101 😉 ), and test it again on your case, it would be very much appreciated. Thanks again for your feedback. PS: @vbargsten : do you want to appear in the contributors section ? If yes, you can tell me if you want a link to your github acount or website and if you want to use your realname or a nickname... |
|
Hi @mbideau, thank you very much for your effort. I am glad to hear from you; I was afraid that something happened, because your activities here were a while back and your domain appeared to be unregistered. I gave it a new try. Unfortunately the program seems to be stuck in a loop. With I'm not sure if this is the same bug showing differently due to the fix. If not we can close it here and open a new issue. |
|
Hi @vbargsten, Wow 120Gb log file ! How long did you let it ran ? A month ? 😅 I have tested it against a moderately large stream file (150Mb) and it ran 24h on a 10 year old i3 computer, generating a 150Mb output log. Could you try to first generate a stream file and tell me about its size (and number of lines that are not metadata changes) ? I have added an argument to the binary to reduce the output, but I didn't had the time to commit it (and I don't have access to that code right now). I'll do it in the coming week I think. Thanks again in advance. |
|
Thank you! A few minutes were enough to generate the 120 GB. I'm not sure if it really repeats or if for every event it just prints the full file tree status. In addition, I've let it run for 2 hours without debug. In fact, the output looks ok. It lists 25000 changes ( mostly caches of MS products :/ ). So, I think it actually works but way too slow - probably a file based diff would be faster. Let me know if you have an idea. |
|
Alright, thank you @vbargsten. The file is way smaller than I thought but bigger than my biggest test file 😃 I have taken the time to commit my code with a less verbose option I guess it would require around 35h on a 10 years old computer to analyze and report all the changes. If it finishes with success on your test data it would be a good sign of its stability, and then I could start to think about how to optimize it 😉 Again, thank you in advance for your well appreciated feedback 🙏 |
|
This is the outcome: I did a quick check what's going on there with If I'm correct the tree isn't printed at all in this mode, right?. So I made the following change: diff --git a/pkg/btrfs-diff.go b/pkg/btrfs-diff.go
index bd1db35..1da67a6 100644
--- a/pkg/btrfs-diff.go
+++ b/pkg/btrfs-diff.go
@@ -166,7 +166,7 @@ func debugInd(ind int, msg string, params ...interface{}) {
// print a diff tree (only if debug mode is enabled)
func (node *nodeInst) PrintTree(w io.Writer, root bool, padding string, lenParent int, indexParent int) {
- if node == nil {
+ if node == nil || !debugMode {
return
}Et voila, now only 3s :) I checked that the output is the same once we remove the timestamps from the line beginning: So the problem is with the tree printing only, otherwise it seems to work as intended. |
|
Awesome @vbargsten ! 🎉 One thing I want to be sure is that no operation is wrongly ignored or wrongly added, so I am going to compare the output to the regular Note: I didn't ran those commands, I just adapted them from another script, may be you'll have to adjust/tweak them. I'll get back to you when I'll have a proper script that does this automatically. You can also PS: thanks also for the perf tool, I didn't know about it, very usefull 🙏 |
|
Hi @mbideau! notes:
Still not completely the same:
It is hard for me to conclude anything from the diff. What in particular should I look for? I created a PR. You can add me as contributor, but I do not insist. My data is now in the git commit log of the PR :) |
|
Thanks again @vbargsten for your well appreciated contributions.
I ran it on my test data (using snapshots made by timeshift on a Manjaro distribution on a spare computer), but I didn't ran into the symlink issue you mention.
😅
Yes, I tracked down a few of those and they all are related to an UPDATE_EXTENT command that doesn't change the file content (used to send "holes"), but
That's not good. There is a code (https://github.com/mbideau/btrfs-diff-go/blob/master/pkg/btrfs-diff.go#L650-L654 and https://github.com/mbideau/btrfs-diff-go/blob/master/pkg/btrfs-diff.go#L721-L725) that do not prints operations on a file if one of the parent directory was renamed (because I assumed a rename precedes a deletion, and if the parent directory was deleted, operations on inner files doesn't matter)... May be I was to bold and there is a bug in there. If by any chance, you could privately send me your test stream file and the output of the If I/we struggle too much to get a proper result in this golang implementation (that I improved over the basis I found), I consider re-using the Thanks again. |
It depends where the symlink points to. If it is absolute, it could point into your current root / subvolume and diff will compare that when not using
Ok, let's ignore the
On further inspection, I can relax that a bit I'd say.
I ran with debug but disabled the PrintTree in the code. Then searched for one of the files. It is detected as deleted, but as you see no Maybe because the parent dirs (/vbargsten/texmf/doc and below) is deleted and added?: I cannot share the full stream metadata, unfortunately. |
|
Hi @vbargsten, |
Seems like it doesn't handle newly created folders. Perhaps folders created anew start with a random name, and then are renamed like the files?
The text was updated successfully, but these errors were encountered: