New feature

It'd be great to be able to get notified (using the TraceObserver) when files are being staged.

Use case

I'm currently building a provenance plugin nf-lamin for logging workflow executions in LaminDB.

It's easy to detect when files are being published (i.e. the outputs of the workflow) using the TraceObserver. However it's not so easy to detect when files are staged (i.e. the inputs of the workflow).

I notice that the nf-prov plugin uses nextflow.prov.util.ProvHelper.getWorkflowInputs() to detect workflow inputs. However, this only allows me to observer the paths after the files have already been staged. That is, I only get to see work/stage-2478a5a8-2313-49c9-8cfe-92ef6483859b/91/ec7b0d3c79c84f8f7e16e07d823a7e/samplesheet-2-0.csv while I actually need https://github.com/nf-core/test-datasets/raw/scrnaseq/samplesheet-2-0.csv.

Suggested implementation

We could modify the TraceObserver. Note that I would add onFileStage and onFileStaging so we could know whether a file is yet to be staged, or whether it has been staged.

trait TraceObserver {
    void onFilePublish(Path destination, Path source)
    void onFilePublishing(Path destination, Path source)
    void onFileStage(Path destination, Path source)
    void onFileStaging(Path destination, Path source)
}

Alternatively, an enum could be added to reduce the number of different functions.

enum FileTransferStatus {
    STARTED,
    FINISHED
}
trait TraceObserver {
    void onFilePublish(Path destination, Path source, FileTransferStatus status) {
        // preserve backward compatibility
        if (status == FileTransferStatus.FINISHED) {
            onFilePublish(destination, source)
        }
    }
    void onFileStage(Path destination, Path source, FileTransferStatus status) {}
}

Happy to send a PR if it helps! :)