Differences between performance metrics between ranger model and its explain

[ManuelSpinola]

I see there are differences for the metrics performance between a ranger model and its explain. Why is that?

aps_ranger <- ranger::ranger(m2.price~., data = apartments, num.trees = 50)

aps_ranger
Ranger result

Call:
ranger::ranger(m2.price ~ ., data = apartments, num.trees = 50)

Type: Regression
Number of trees: 50
Sample size: 1000
Number of independent variables: 5
Mtry: 2
Target node size: 5
Variable importance mode: none
Splitrule: variance
OOB prediction error (MSE): 93997.56
R squared (OOB): 0.8856602

exp_ranger <- explain(aps_ranger, data = apartments, y = apartments$m2.price)

mod_per <- model_performance(exp_ranger)

mod_per
Measures for: regression
mse : 21442.99
rmse : 146.4342
r2 : 0.9738904
mad : 84.57676

Residuals:
0% 10% 20% 30%
-491.535000 -160.125376 -109.455093 -75.266200
40% 50% 60% 70%
-43.538133 -20.939611 8.850819 47.124000
80% 90% 100%
97.417832 197.996033 759.546667

[mayer79]

The metrics of ranger are calculated from out-of-bag predictions, those of DALEX on the data provided.

Note that it basically never makes sense to study random forest performance (non-oob) on the training data except to demonstrate its massive overfit.

[ManuelSpinola]

Thank you very much Michael. Manuel El mar, 5 mar 2024 a las 0:21, Michael Mayer ***@***.***>) escribió:

…

The metrics of ranger are calculated from out-of-bag predictions, those of DALEX on the data provided. Note that it basically never makes sense to study random forest performance (non-oob) on the training data except to demonstrate its massive overfit. — Reply to this email directly, view it on GitHub <#561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFI3FB7DQ3XB2RAAPZEGX2LYWVP4XAVCNFSM6AAAAABEGH3HVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGA2DENZWGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- *Manuel Spínola, Ph.D.* Instituto Internacional en Conservación y Manejo de Vida Silvestre Universidad Nacional Apartado 1350-3000 Heredia COSTA RICA ***@***.*** ***@***.***> ***@***.*** Teléfono: (506) 8706 - 4662 Sitio web institucional: ICOMVIS <http://www.icomvis.una.ac.cr/index.php/manuel> Sitio web personal: Sitio personal <https://mspinola-sitioweb.netlify.app> Blog sobre Ciencia de Datos: Blog de Ciencia de Datos <https://mspinola-ciencia-de-datos.netlify.app>