[R] Use inplace predict #9829

david-cortes · 2023-11-30T21:00:35Z

This PR switches the predict method to use inplace prediction when possible (dense matrix / CSR matrix as inputs).

It seems that there are quite a lot of limitations around when it is or isn't possible to use these functions which are not documented in the readthedocs page, so I am not sure if the conditions that I am adding here to check for whether inplace predict is supported are correct.

I'm also not sure what's the most efficient way of dealing with all the JSON manipulation here.

Along the way, since it is refactoring some of the earlier code introduced for processing the "missing" parameter, it also modifies the function that creates DMatrix objects from CSR/CSC. It doesn't make a practical difference there though, since currently only classes inheriting from "DsparseMatrix" (float64 values) are supported, but might come handy in the future if other sparse matrix classes (like "LsparseMatrix") become supported in the future.

david-cortes · 2023-11-30T21:02:03Z

ref #9810
fixes #7616

david-cortes · 2023-12-01T17:12:15Z

I see there's a failing MSVC job, but it doesn't seem to contain any test failure, only warnings. No idea what went wrong there.

trivialfis · 2023-12-01T23:59:36Z

Always fun to have R failing on Windows when one thinks everything is ready. ;-) I restarted the CI and it seems to be failing again. I can help debug it on Monday next week.

david-cortes · 2023-12-06T18:17:33Z

Let's hope the R update in the msvc job helps here.

trivialfis · 2023-12-11T01:42:57Z

Let me try to reproduce it.

trivialfis · 2023-12-11T06:17:56Z

R-package/R/xgb.Booster.R

+  use_as_csr_matrix <- FALSE
+  inplace_predict_supported <- !predcontrib && !predinteraction && !predleaf && is.null(ntreelimit)
+  if (inplace_predict_supported) {
+    if (config$learner$learner_train_param$booster != "gbtree") {


dart is supported as well. Only gblinear is not supported.

trivialfis · 2023-12-11T07:42:53Z

I can reproduce the error on my machine, and somehow magically got it working by cout << res_code << endl in the new prediction function. The log from MSVC looks suspicious.
Other than the error, the prediction function needs to consider base margin as well, which requires the use of proxy DMatrix. I will look into the QDM PR for the proxy implementation in R.

 C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.37.32822\include\stdarg.h(29,9): warning C4068: unknown pragma 'Rf_warning' [D:\a\xgboost\xgboost\build\R-package\xgboost-r.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.22621.0\ucrt\corecrt_malloc.h(13,9): warning C4068: unknown pragma 'Rf_warning' [D:\a\xgboost\xgboost\build\R-package\xgboost-r.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.22621.0\ucrt\corecrt_malloc.h(14,9): warning C4068: unknown pragma 'Rf_warning' [D:\a\xgboost\xgboost\build\R-package\xgboost-r.vcxproj]

david-cortes · 2023-12-11T18:51:08Z

Stupid question: wouldn't setting a proxy dmatrix inside the prediction function be slower and/or consume the same amount of memory as creating a dmatrix to use for predict instead? Or am I misunderstanding the mechanism?

david-cortes · 2023-12-11T20:49:54Z

BTW, while trying to add base margin here, I ran into an issue which manifested itself with both inplace predictions and dmatrix predictions: #9869

trivialfis · 2023-12-12T03:01:14Z

wouldn't setting a proxy dmatrix inside the prediction function be slower and/or consume the same amount of memory

No, the proxy DMatrix is just a reference to the original data, so, don't free the data without first freeing the proxy dmatrix ...

trivialfis · 2023-12-12T03:09:14Z

With hindsight, the inplace prediction can be completely implemented with proxy DMatrix without those special parameters for csr/dense, but at this point, there's no real benefit to making such breaking changes.

david-cortes · 2023-12-12T19:27:20Z

Added base_margin as a parameter to predict, plus a function to use the in-place prediction on data frames.

david-cortes · 2023-12-12T21:02:10Z

@trivialfis After adding support for categorical features for data frames, I just realized that passing them to predict will now require re-identifying which of those earlier factor columns were categorical, given that they need to be manually converted to base-0 indexing.

What would be the best way to retrieve that information (which columns are categorical) from the booster?

trivialfis · 2023-12-13T00:43:48Z

@david-cortes In Python, the booster has the same feature_type parameter, search XGBoosterGetStrFeatureInfo in core.py.

david-cortes · 2023-12-17T22:05:54Z

@trivialfis Thanks for the tip. Would be very helpful if you could investigate the failing MSVC job here so that we can merge this PR soon, as I'm finding some of the code here will be needed for the QuantileDMatrix and other PRs too.

I unfortunately don't know how best to debug an MSVC-compiled module that needs to be imported in R, as it can't use typical tools like sanitizers with a regular R build.

trivialfis · 2023-12-19T02:49:21Z

I can try, but I admit I don't use Windows very often.

hcho3 · 2023-12-20T06:24:50Z

Is this PR independent of #9902?

I use Windows as a daily driver, so should be able to help debug the error in this PR.

trivialfis · 2023-12-20T06:28:47Z

Is this PR independent of #9902?

yes.

trivialfis · 2024-01-02T18:54:32Z

The compiler error with sanitizer was about a different file in XGB (refresher), I will try again tomorrow.

trivialfis · 2024-01-15T09:30:14Z

I have made no progress on the Windows error, we will have to delay this PR for a while.

trivialfis · 2024-01-16T08:56:37Z

Discussion with @hcho3 , we are likely to move away from MSVC for the R package, along with which, we will drop support for building R package with GPU on Windows using MSVC. We haven't ruled out the possibility of using rtools and CMake to build GPU support, in addition, recommending WSL2 is a potential option as well.

david-cortes · 2024-01-16T17:38:16Z

Discussion with @hcho3 , we are likely to move away from MSVC for the R package, along with which, we will drop support for building R package with GPU on Windows using MSVC. We haven't ruled out the possibility of using rtools and CMake to build GPU support, in addition, recommending WSL2 is a potential option as well.

Is that just for the R package, or also for the python package for windows? If just for R, have there been MSVC-specific issues like this before? Curious about what makes it problematic.

trivialfis · 2024-01-16T18:16:26Z

It's just the R package,mostly because I couldn't figure out the compiler warning and the error in this PR. So far, it seems MSVC is not handling the C headers from R very well.

trivialfis · 2024-01-20T16:51:19Z

Could you please help rebase the PR?

david-cortes · 2024-01-20T19:59:36Z

Updated, but I'm now not 100% sure if everything is correct.

I suddenly started getting an error in this test and the one right after it after merging the latest master:

xgboost/R-package/tests/testthat/test_basic.R

Line 142 in c5d0608

    
           expect_true(all(matrix(pred_by_train_0, byrow = TRUE) == matrix(pred_by_xgboost_0, byrow = TRUE)))

Before, it was using ntreelimit, which I guess meant that the code was converting to DMatrix, while now it's actually using inplace predict.

It looks like predictions from in-place match those from DMatrix and from xgb.train up to only about ~1e-7 in this particular case (dart booster), while elsewhere they seem to match closer.

BTW, would be ideal if you could merge the PR about data iterators before this one:
#9913

trivialfis · 2024-02-18T20:15:13Z

Updated, but I'm now not 100% sure if everything is correct.

Please let me know how things are going. ;-) Is there anything I can help?

david-cortes · 2024-02-19T18:04:05Z

Updated, but I'm now not 100% sure if everything is correct.

Please let me know how things are going. ;-) Is there anything I can help?

PR is ready for review. Would be ideal if you could look into this issue with loss of floating point precision for dart. Since the difference between inplace and dmatrix is below $10^{-6}$ I guess it might not be a correctness problem, but it's odd that it gets a larger difference with dart than with gbtree.

R-package/R/xgb.Booster.R

trivialfis · 2024-02-22T16:39:26Z

R-package/R/xgb.Booster.R

+#'              encoding (same as during DMatrix creation) - hence, one should not pass a `factor`
+#'              under a column which during training had a different type.
+#'        }
+#' @param missing Float value that represents missing values in data (e.g., 0 or some other extreme value).


We usually use nan as missing value, since XGBoost can handle 0 just as any other real number.

This is also a headache for us since many sparse matrix implementations assume 0 as missing.

Yes, default value is R's NA, which gets translated to either NAN or R_NaInt depending on the input type.

R-package/R/xgb.Booster.R

trivialfis · 2024-02-22T16:42:07Z

R-package/R/xgb.Booster.R

+  n_row <- NULL
+  if (!is_dmatrix) {
+
+    inplace_predict_supported <- !predcontrib && !predinteraction && !predleaf


We should be able to support leaf in the near future.

trivialfis · 2024-02-22T16:43:52Z

R-package/src/xgboost_R.cc

+
+struct ProxyDmatrixError : public std::exception {};
+
+struct ProxyDmatrixWrapper {


is this related to the R implementation?

Yes, it's used in order to break out of a C++ block, so that C++ objects get destructed before throwing an R error.

trivialfis · 2024-02-23T16:10:41Z

Thank you for the work on inplace prediction! The R package is looking much better now thanks to your great work.

use inplace predict

2471933

simplify further

b605ef7

david-cortes added 2 commits December 2, 2023 20:21

solve merge conflicts

af81946

solve merge conflicts

5b8c661

david-cortes mentioned this pull request Dec 10, 2023

[Draft] [R] Add QuantileDMatrix creation from dense matrices #9864

Closed

trivialfis reviewed Dec 11, 2023

View reviewed changes

enable inplace_predict also for non-training dart

080f372

david-cortes added 4 commits December 12, 2023 19:06

add base_margin to predict

d1fdac0

solve merge conflicts

bd62c5b

add test for base_margin

9a19e3d

add inplace prediction also for data frames

d07162b

corrections for prediction on data.frame

dea90b0

linter

9347a41

hcho3 and others added 4 commits January 2, 2024 13:24

Merge remote-tracking branch 'upstream/master' into inplace_predict

50811c3

solve merge conflict

7a775d0

solve conflicts

2b32b4f

update roxygen

8584093

trivialfis removed their assignment Jan 15, 2024

hcho3 mentioned this pull request Jan 17, 2024

[CI] Skip MSVC when building R package #9995

Merged

solve conflicts

5fec492

david-cortes added 7 commits January 30, 2024 21:37

solve merge conflicts

5b045ba

use anonymous namespace instead of statics

4f07239

linter

218c3db

solve merge conflicts

b999c2a

Merge branch 'master' into inplace_predict

afde244

add inplace predict also for sparse vectors

e03527c

update docs

16cb633

trivialfis reviewed Feb 22, 2024

View reviewed changes

david-cortes added 3 commits February 22, 2024 19:07

Merge branch 'master' into inplace_predict

97e7e76

update wording

b7ca893

move warning to error

9609b61

trivialfis approved these changes Feb 23, 2024

View reviewed changes

trivialfis merged commit f7005d3 into dmlc:master Feb 23, 2024
31 of 32 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[R] Use inplace predict #9829

[R] Use inplace predict #9829

david-cortes commented Nov 30, 2023

david-cortes commented Nov 30, 2023

david-cortes commented Dec 1, 2023

trivialfis commented Dec 1, 2023

david-cortes commented Dec 6, 2023

trivialfis commented Dec 11, 2023

trivialfis Dec 11, 2023

trivialfis commented Dec 11, 2023

david-cortes commented Dec 11, 2023

david-cortes commented Dec 11, 2023

trivialfis commented Dec 12, 2023

trivialfis commented Dec 12, 2023

david-cortes commented Dec 12, 2023

david-cortes commented Dec 12, 2023

trivialfis commented Dec 13, 2023

david-cortes commented Dec 17, 2023 •

edited

Loading

trivialfis commented Dec 19, 2023

hcho3 commented Dec 20, 2023

trivialfis commented Dec 20, 2023

trivialfis commented Jan 2, 2024

trivialfis commented Jan 15, 2024

trivialfis commented Jan 16, 2024

david-cortes commented Jan 16, 2024

trivialfis commented Jan 16, 2024

trivialfis commented Jan 20, 2024

david-cortes commented Jan 20, 2024

trivialfis commented Feb 18, 2024

david-cortes commented Feb 19, 2024

trivialfis Feb 22, 2024

david-cortes Feb 22, 2024

trivialfis Feb 22, 2024

trivialfis Feb 22, 2024

david-cortes Feb 22, 2024

trivialfis commented Feb 23, 2024


		struct ProxyDmatrixError : public std::exception {};

		struct ProxyDmatrixWrapper {

[R] Use inplace predict #9829

[R] Use inplace predict #9829

Conversation

david-cortes commented Nov 30, 2023

david-cortes commented Nov 30, 2023

david-cortes commented Dec 1, 2023

trivialfis commented Dec 1, 2023

david-cortes commented Dec 6, 2023

trivialfis commented Dec 11, 2023

trivialfis Dec 11, 2023

Choose a reason for hiding this comment

trivialfis commented Dec 11, 2023

david-cortes commented Dec 11, 2023

david-cortes commented Dec 11, 2023

trivialfis commented Dec 12, 2023

trivialfis commented Dec 12, 2023

david-cortes commented Dec 12, 2023

david-cortes commented Dec 12, 2023

trivialfis commented Dec 13, 2023

david-cortes commented Dec 17, 2023 • edited Loading

trivialfis commented Dec 19, 2023

hcho3 commented Dec 20, 2023

trivialfis commented Dec 20, 2023

trivialfis commented Jan 2, 2024

trivialfis commented Jan 15, 2024

trivialfis commented Jan 16, 2024

david-cortes commented Jan 16, 2024

trivialfis commented Jan 16, 2024

trivialfis commented Jan 20, 2024

david-cortes commented Jan 20, 2024

trivialfis commented Feb 18, 2024

david-cortes commented Feb 19, 2024

trivialfis Feb 22, 2024

Choose a reason for hiding this comment

david-cortes Feb 22, 2024

Choose a reason for hiding this comment

trivialfis Feb 22, 2024

Choose a reason for hiding this comment

trivialfis Feb 22, 2024

Choose a reason for hiding this comment

david-cortes Feb 22, 2024

Choose a reason for hiding this comment

trivialfis commented Feb 23, 2024

david-cortes commented Dec 17, 2023 •

edited

Loading