fsync_data in SyncPolicy?

[fulmicoton]

Right now mrecordlog never explicitly calls fsync_data.

One tricky thing is that in Quickwit we want to be able to fsync_data only once per ingest batch and one ingest batch can target several queues at the same time.

We need to have an API that makes that possible. One tricky part is that this is not just one file, it is all of the files that have been touch since the last sync.

If there are no pretty way to do all that, as a simplfiication we could fsync_data when we finish a file, and just add a method to fsync the tail file or something like that. (fsync_data is sufficient)

Also, we probably should fsync files upon their creation as well as their parent.

[fulmicoton]

@trinity-1686a Do you remember any discussion we may have had around fsync? I can't recall anything but I assume this choice was made to cut corners, as it is the default for rocksdb and leveldb.

[trinity-1686a]

I thought we did fsync on flush, but apparently no, we just flush and hope for the best 😬
I don't recall a discussion on the subject either

[trinity-1686a]

fixed by #60