Rework windows "Physical size" calculation, added Unit Tests #9
Conversation
* Manually implemented windows File Namespace name generation * added some unit tests closes scullionw#5
Current implementation provides more correct size for files of size around 2k and would show it as 4k physical size. CompressedFileSize would report smaller size. But it still have problems of being inconsitent with Windows explorer for files smaller than 512 bytes. Such small files are actually stored inside directory, not in own clusters, and explorer reports them as 0. Actual threshold is not 512, it's dynamic, for now I don't know how to detect it. Also this algorithm ignores alternate data streams. Same for previous implementation. To be more precise -- alternate data streams can also be compressed, as I learned today.
…s for windows Use COMPRESSION_FORMAT_LZNT1 instead of COMPRESSION_FORMAT_DEFAULT, perhaps it would help on WinServer
Mart-Bogdan
force-pushed
the
max-path
branch
from
452bbae to
d162b0e
Compare
…version as it was accidentally increased twice, skiping one version number.
|
It has lots of clippy warnings, but I think they should be resolved by separate PR. And then we can even reject PRs that don't stratify clippy. |
|
I have written simple benchmark, and it shows that this solution is even faster: https://github.com/Mart-Bogdan-TMP/dirstat-rs-benchmark/actions/runs/3504907900/jobs/5870888010 But we need to take this tests with grain of salt. It only measures functions/syscall overhead. Filesystem cache takes most on itself. (if we want to exclude FS cache from equation, we would need another testing methodology, that would flush caches. It's possible, at least on Linux). On my local machine I've got following results: |
|
Hello, @scullionw. Can you please review my PR? |
|
Hi! Thanks for the PR! Sorry about the delays, quite busy these days. This is a pretty big one and I can't review it all right now. However, if you want to split the PR (tests, workflows, changes to windows impl), we can merge the first two! |
|
Hi. Could try it till Friday. P.S. wanted to work on other stuff regarding this app as well. |
|
|
|
I have created PR #10 with tests |
Added:
Current implementation provides more correct size for files of size around 2k
and would show it as 4k physical size. CompressedFileSize would report smaller size.
But it still have problems of being inconsistent with Windows explorer for files smaller than 512 bytes.
Such small files are actually stored inside directory, not in own clusters, and explorer reports them as 0.
Actual threshold is not 512, it's dynamic, for now I don't know how to detect it.
Also this algorithm ignores alternate data streams. Same for previous implementation.
To be more precise -- alternate data streams can also be compressed, as I learned recently.
Main difference: we are using syscall that uses file handle (that we already have opened) instead of getting by file path.
And getting file path for long path required reformatting path into special format. Rust's std File is performing this for us under the hood. I first tried to fix this, and format string with
\\.\prefix (for network path UNC prefix) to enable long path, but that's big tricky function that have to be maintained, and usingGetFileInformationByHandleExgives us more flexibility for further improvements.