[Filebeat] possible filestream performance regression on Windows

[andrewkroh]

It looks like on Windows the filestream input calls OpenFile, GetFileInformationByHandle, and CloseFile twice for each log line read (six syscalls per line read). It would be good to get a second opinion on this. The code path is through:

beats/libbeat/reader/readfile/metafields.go

Line 64 in e4d287f

err = setFileSystemMetadata(r.fi, message.Fields)

beats/libbeat/common/file/file_windows.go

Lines 44 to 48 in 8ff6894

	func GetOSState(info os.FileInfo) StateOS {
	// os.SameFile must be called to populate the id fields. Otherwise in case for example
	// os.Stat(file) is used to get the fileInfo, the ids are empty.
	// https://github.com/elastic/beats/filebeat/pull/53
	os.SameFile(info, info)

https://pkg.go.dev/os#SameFile

https://cs.opensource.google/go/go/+/master:src/os/types_windows.go;l=243-249?q=loadFileId&ss=go%2Fgo

Relates: #36065

[rdner]

@andrewkroh thanks for discovering this, it comes to me as a surprise that os.SameFile opens and closes files on Windows, furthermore, it actually acquires a file system lock doing so.

The problem is this loadFileId function here:

https://cs.opensource.google/go/go/+/refs/tags/go1.21.4:src/os/types_windows.go;l=165-214

used by os.SameFile:

https://cs.opensource.google/go/go/+/refs/tags/go1.21.4:src/os/types_windows.go;l=231-241

As the comment states, this call is required on Windows to fill the IDs in:

fs.vol = i.VolumeSerialNumber
fs.idxhi = i.FileIndexHigh
fs.idxlo = i.FileIndexLow

I believe our code populates the IDs elsewhere, so we should be able to get rid of this os.SameFile call in setFileSystemMetadata by storing the values once they are set for the first time.

I'll have a deeper look as soon as I can.

@pierrehilbert we should prioritise this.

[pierrehilbert]

This is currently in our next sprint.