[Question] Progress during inference

[simpsus]

When predicting with a large booster and a large dataset, the inference can take several hours (at least on my setup). It would be nice if there was the possibility to see a progress.
Ideally I would love to have a tqdm bar of the number of samples.

I am willing to accept some slowdown for the information, but putting my samples in chunks and then calling predict on each chunk wrapped by a tqdm ... I hope that there is some better way.

Thank you!

[jameslamb]

Thanks for using LightGBM, and taking the time to put up this suggestion.

I personally don't support taking on tqdm as a dependency of this project for this purpose. I understand you might be willing to accept the overhead it introduces in a throughput-sensitive application (large number of samples, batch predictions), but I don't think others who call predict() in latency-sensitive applications (small number if samples, on-demand scoring) would be happy about it.

See #5867 for some related discussion.

Some things you could try to speed up predictions in the situation you've described:

• pass num_threads > 1 through params in predict(), to take advantage of multithreading
• use the lightgbm.dask interface to split the work of generating predictions over multiple machines
• pass pred_early_stop through parameters to stop the prediction process once it seems that adding output from later trees isn't changing the predictions much

Docs on parameters: https://lightgbm.readthedocs.io/en/latest/Parameters.html

Docs on how to use lightgbm.dask: https://lightgbm.readthedocs.io/en/latest/Parameters.html

[simpsus]

thank you for the insights, closing this issue

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.