Performance information

[robinjournet]

After perusing the API documentation, I saw there was a method to obtain run time by using run_start & run-stop.
This allows one to already have some idea on the elapsed time for a whole gate worker process.
By using a list of documents and running a pipeline on one document at a time, it is possible to obtain the time taken by each document.

I would like to go further and be able to obtain the time taken by each PR. It is indeed possible to obtain a list of the pipelines' PR using the .getPrNames(). Knowing this, which methods of python gate NLP one can leverage to obtain the elapsed time when switching from one PR to another while processing a document with a pipeline ?

Logging allows recording the main steps but don't go deep enough for my use case. (benchmarking and documents annotations viewer)

Worker run: get known document names
Worker run: save document to 13787.xml mimetype:
Worker run: remove resource13787.xml_00019
Worker run: load document from C:\\Users\\robin\\Documents\\Xxxx\\SSDD\\Gate\\Enumeration\\Corpus_6\\15056.xml mimetype:
Worker run: run controller ListAPuce for 15056.xml_0001A

Furthermore, would it be even possible to obtain pieces of information on each jape grammars when several are nested in one jape grammar( multi phase jape file) without having to resort to adding line to the original jape grammars ?

Thank you for your advices and opinions on this matter

[johann-petrak]

The way you use gatenlp here this is more a question about Java GATE really: if I understand correctly you run everything in Java and only exchange the result back to python at some point. That is perfectly fine, but there is nothing gatenlp can do about how Java GATE runs a pipeline.

Java GATE does have some support for profiling processing resources: https://gate.ac.uk/userguide/chap:profiling

Not that the run_start and run_stop functions are not meant for profiling, just to give an idea for how long a whole program runs by logging the times. This is meant to be used in "main" code of CLI programs mainly. However Python has a lot of direct easy support for profiling runtimes.

In order to profile PRs instead of the whole pipeline from Python, you would need to call each PR individually. The GateWorker currently does not have direct support for this, but you can basically execute almost any Java command from Python through the jvm attribute of the GateWorker and this should allow you to call each PR individually.

In general, I would recommend though to analyse performance of your Java GATE pipeline from Java GATE directly and only then check what the overhead of running the Java Pipeline and exchanging the documents with Python is.