FBM compatibility with LightGBM

[nbenn]

I'm trying to use an FBM matrix for training a LightGBM model (could probably also be done for xgboost in similar fashion). For this I think it's easiest (for dependencies sake) to pass a pointer to start of the data via R as an external pointer. I hacked something together:

https://github.com/nbenn/bigstatsr/blob/b668f2fdcfafbf1d280250bca9ca0cabaa88a855/src/FBM-xptr.cpp#L27-L35

How do you feel about this?

See also microsoft/LightGBM#2184.

[privefl]

I mean, if it is as easy as that, why not.
I don't know about LightGBM, but XGBoost handles passing an external pointer? Is that new?

[privefl]

Any update on this?

[privefl]

Any update on this?

[nbenn]

I'm sorry, I guess we can close this for now. As long as there is no support on the LightGBM side, there's no need for a getDataPtrFBM() function. While my idea did make the feature request list at microsoft/LightGBM#2302, there hasn't been much movement there.

[privefl]

Thanks for following up.

Just to make sure: it is on their side that some work needs to be done? Not on my side?

[nbenn]

Yeah, so from what I remember, for this to work out, both a function for getting a pointer to the data as EXTPTRSXP and a way for LightGBM to make use of this pointer is needed. I put together a quick suggestion for them how this could be done, but never really got any feedback. So I guess, they're not really that interested (or it turns out to be move involved than I understand).