Support for enabling the call to single row predictions in parallel way

[tarunreddy1018]

Currently, LGBM_BoosterPredictForMatSingleRowFast, LGBM_BoosterPredictForMatSingleRow were meant to do predictions for a single row of input. So, with reference to this issue #3751 raised few months back regarding the thread safety issues in these methods when they were called in a parallel way. I would like to request if there are any plans to either make these methods to also support parallel predictions (or) have a completely new method which is meant for single row predictions but it can be called in a parallel way. I believe that there is still a scope to make this method to be called in a parallel way. As far as I have gone through the source code it seems like this Predictor object is something which gets used for each call. Creating a new Predictor object for each function call might not be an ideal way to go, because that can increase the memory consumption. But if there is a way to pool these Predictor objects (or) create a fixed size pool of these Predictor objects so that it can be reused for future calls, I believe that we can have a check on memory consumption along with achieving parallelism.

If this can be done then it can be used in both ways. Either normally or in a parallel way. This can be a great improvement in terms of latencies when this is used in production environments.

Could you please let me know if there are any plans in future in terms of improving the parallel predictions.

[StrikerRUS]

I guess this is the same as #4290.

[no-response]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!