Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The evaluation of ensemble model performance and the calculation of environmental variable importance within the ensemble model #11

Open
xunuan2020 opened this issue Apr 26, 2024 · 0 comments

Comments

@xunuan2020
Copy link

Hello, SIR:
This question is about how to evaluate ensemble model performance and calculate environmental variable importance.
When conducting the following steps, I utilized the parameters below:
mxnt.mdls.preds.lst<-calib_mdl_b(ENMeval.o.l = ENMeval.res.lst,
a.calib.l = clim.stack.list,
occ.l = occ.list,
mSel = "EBPM")
I obtained the following results in model performance and variable importance:
(1) metric.var.PermImportance.csv
sp sel.crit wc2_13 wc2_14 wc2_15 wc2_3 wc2_4 wc2_8 wrb_MostProba
1 R Mod.EBPM_1 3.3966 0 0 3.5917 17.5038 20.1878 18.6949
2 R Mod.EBPM_2 4.4336 1.446 2.0736 3.4377 16.4984 1.3751 45.5691
3 R Mod.EBPM_4 3.1912 9.8249 14.111 2.0451 5.4621 1.5338 36.9477
4 R Mod.EBPM_5 1.7422 2.8706 0 3.5272 11.5683 0.1122 59.874
5 R Mod.EBPM_3 2.4887 23.3218 1.7355 0 2.6066 0.0851 51.2607
(2) sel.mdls.smmr.Rho.csv
Optimality criteria FC RM AICc delta AICc wAIC parameters Rank AIC avg OR10 avg ORLPT avg AUC
2 EBPM_1 LQ 0.5 NA NA NA 17 12 0.25 0.083333333 0.986241667
10 EBPM_2 LQH 1 NA NA NA 29 20 0.416666667 0.083333333 0.983875
11 EBPM_4 LQHP 1 NA NA NA 31 21 0.416666667 0.083333333 0.983333333
16 EBPM_5 LQH 1.5 NA NA NA 21 25 0.333333333 0.083333333 0.982458333
18 EBPM_3 LQHPT 1.5 NA NA NA 23 27 0.333333333 0.083333333 0.983758333
But I want to obtain the overall evaluation results of this ensemble model, not individual evaluation results. What should I do?
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant