RandomAccess -- OK at gathering, not so good at scattering

[rmsimpson]

Although the new RandomAccess read/write patterns are a welcome extension, there's a couple of overloads that would really make them shine.

• The multi-buffer read/write methods allow you to pass in an array of memory buffers, but not an array of offsets. This really limits the usefulness of the functions, and they just become a multi-buffer sequential read/write. Given the limits, why not just pass in a larger buffer in the first place?

• Multi-buffer read/write functions don't return until all buffers are full. One of the most performant ways to read data from a source is to always keep a read operation queued at all times, so the OS always has something to do while you're doing something else. A multi-buffer read operation is perfect for that, but only if you can not only specify an array of offsets, but also get notified when each buffer fills up so you can then re-queue one or more new buffers to ensure the OS stays busy.

I don't have any one particular design in mind for changing the functions. Maybe pass in an IReadOnlyList<> of offset/memory pairs, or pass in two lists, or whatever floats your design boat.

As for notification, maybe pass in a callback Func<> or Action<> Some way to get notified as each buffer gets filled would be amazing.

Where this seems to have the most benefit is on network file transfers. I'm specifically talking about 10gbit ethernet, where opening two FileStreams (source remote, target local) and calling CopyTo() just baaaarely tops at about 1.8gbps on a good day.

Similarly calling a powershell function like Get-FileHash on a network file can just top out at about 3gbps which I assume just uses FileStream under the hood.

I wrote a test implementation of a copy/reader using RandomAccess for async reads and writes with cycling buffers. I was able to roll the same copy and hash functionality and hit 9gbps. I could've rolled a much simpler version of my code if multi-buffer reads and writes had the above features!

[svick]

The multi-buffer read/write methods allow you to pass in an array of memory buffers, but not an array of offsets.

Those overloads take (ReadOnly)Memory<byte>. And that can represent an offset into a buffer, e.g. by using array.AsMemory(offset).

[rmsimpson]

Each Read/Write version (async and non-async) has 2 overloads. One takes a single Memory<byte>, the other takes an IReadOnlyList<Memory<byte>>

I'm referring to the one that takes a IReadOnlyList of memory blocks, but only accepts a single offset.

Also, I'm not referring to the offset inside the memory block, I'm referring to the offset at which to read in the file.

Since the Read/Write functions accept an array of memory blocks but only one single offset in the file from which to fill those memory blocks, the function has limited usefulness.

What I'm interested in, is passing in say 4 Memory buffers, and 4 corresponding FILE OFFSETS from which to fill those 4 buffers.

[rmsimpson]

I realize though, looking at the Win32 API's that .NET is following ReadFileScatter()'s call, which doesn't provide that kind of functionality.