Support time_gran and max_pages fuse options #239
Conversation
This updated the supported FUSE protocol from 7.12 to 7.28. The time_gran field can be set to a multiple of one nanosecond, if the timestamps of files has less than nanosecond precision. This avoids mismatches when the kernel compares timestamps. The max_pages feature raises the maximum size of a FUSE message from 32 to 256 pages on Linux >= 4.20. This affects both reads and writes. To be able to read more than 128 KB in a single ReadOp, Direct I/O must also be used, since the kernel otherwise splits the read into 128 KB chunks and requests them asynchronously (if max_readahead is at least 128 KB, otherwise max_readahead is used, or 4 KB if it is disabled).
|
Would you mind also adding a mount option to set the max pages? It just makes it easier to test the difference caused due to increasing the max pages. |
|
Thank you, this looks interesting. I'm traveling for about a week but will get back to testing this after that. |
|
@tv42 Thanks for the update, no rush! |
|
@tv42 One think I'd like to discuss is wether the allocation of the buffer pool for fuse messages needs to be changed, because this change causes each buffer to be allocated with 1 MB + 4K instead of 128 K + 4 K. In my testing I didn't really see a notable jump in memory usage, but that might be different with async reads or writes when lots more fuse messages can be in flight. |
|
@tv42 @felixbuenemann |
|
@ivandeex No idea, but since other features with a higher fuse level have been implemented in the mean time this will need to ne rebased on top of those changes. I have also started work to support macFuse 4.x, but I currently don't have the time to finish it. I could share my findings, if rclone is working on that as well (will be required for arm64 (M1) Macs). |
@tv42 cc @darthShadow |
|
Yeah... the devil is in the details, in this case mostly the interaction with the memory allocation. It's already the part that slows down filesystems most. |
|
Ah, alright. I am not very familiar with the working of fuse systems in general so can't comment on the impact of this change in terms of memory. However, I am willing to help out with any testing at different values of max pages, if needed. |
This updated the supported FUSE protocol from 7.12 to 7.28.
The time_gran field can be set to a multiple of one nanosecond, if the
timestamps of files has less than nanosecond precision. This avoids
mismatches when the kernel compares timestamps.
The max_pages feature raises the maximum size of a FUSE message from 32
to 256 pages on Linux >= 4.20. This affects both reads and writes.
To be able to read more than 128 KB in a single ReadOp, Direct I/O must
also be used, since the kernel otherwise splits the read into 128 KB
chunks and requests them asynchronously (if max_readahead is at least
128 KB, otherwise max_readahead is used, or 4 KB if it is disabled).