--fix-padding requires 2 passes for stabilizing the output

[connesc]

I'm not sure it's a real issue.

My understanding of the --fix-padding flag is that it normalizes the created NSP/NSZ instead of preserving the original padding.
So, once this normalization has been done a first time, I would expect --fix-padding to become a no-op.

This is not what I observe in my tests with v4.6.1:

# Nominal case without --fix-padding
nsz -C --rm-source test.nsp
nsz -D --rm-source test.nsz
# Resulting test.nsp is identical to original test.nsp, good

# Unexpected behavior with --fix-padding
nsz -C --rm-source --fix-padding test.nsp
nsz -D --rm-source test.nsz
# Resulting test.nsp is different from original test.nsp, expected
nsz -C --rm-source --fix-padding test.nsp
nsz -D --rm-source test.nsz
# Resulting test.nsp is different from previous test.nsp, unexpected
nsz -C --rm-source --fix-padding test.nsp
nsz -D --rm-source test.nsz
# Resulting test.nsp is identical to previous test.nsp, expected

Note that this is the same when --fix-padding is used during decompression steps only, or mixed between compression and decompression steps: the resulting NSP changes twice then stabilizes forever.

This is actually easy to work around, but I would like to be sure it's not hiding something bigger.

[connesc]

Hi @nicoboss!
Would you have any idea on what is happening behind the scene?
I'm about to recompress all my NSZ files before writing them to a long-term storage medium. Thus, I would like my files to be as clean as possible (i.e. normalized and bug free).
So, I wonder if I should use the result of the first or the second pass.
Thanks!

[nicoboss]

Hi @nicoboss! Would you have any idea on what is happening behind the scene? I'm about to recompress all my NSZ files before writing them to a long-term storage medium. Thus, I would like my files to be as clean as possible (i.e. normalized and bug free). So, I wonder if I should use the result of the first or the second pass. Thanks!

Sorry for not looking into this earlier. Almost nobody is using --fix-padding which is why I didn't saw this issue as that urgent. I will look into it in the next few days and hopefully soon fix this.

If you want to see what's happening behind the scenes you can always do so yourself using HexCmp2 from Fairdell Software or any other hex editor with a diff function. HexCmp2 compare the result of running --fix-padding once with running it twice. If you do, please let me know about your findings as knowing what's going wrong will obviously help fixing this issue.

[connesc]

Here is what I get when comparing the outputs of first pass and second pass with the good old VBinDiff.

Title 1

Title 2

It looks like second pass is adding 32 bytes of padding. After that point, the rest of the file is identical.
I hope it can help! 😃

[nicoboss]

Here is what I get when comparing the outputs of first pass and second pass with the good old VBinDiff.
Title 1
Title 2

It looks like second pass is adding 32 bytes of padding. After that point, the rest of the file is identical. I hope it can help! 😃

Thanks a lot for providing the diff. Sorry for the late replay. I was on ski holiday during the last week. The first pass seams to produce an output matching the No-Intro specifications while the second pass adds some useless padding. --fix-padding is not supposed to change anything if the file is already No-Intro standard compliant. This likely is caused by it somehow thinking that that the game has zero padding and so it adds an entire 0x20 block of padding while there already is padding. I have the feeling that it somehow thinks that the padding is part of the FileEntryTable. I will soon investigate this. This confirms previous suspicious that #150 is still not fully fixed. Let's reopen it and treat them as combined issues. I’m really sorry that --fix-padding is still broken..