Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loo_R2 and further arguments passed to ‘posterior_epred’ and ‘log_lik’ #1711

Open
avehtari opened this issue Nov 27, 2024 · 1 comment
Labels
Milestone

Comments

@avehtari
Copy link
Contributor

avehtari commented Nov 27, 2024

loo_R2() allows "Further arguments passed to ‘posterior_epred’ and ‘log_lik’,
which are used in the computation of the R-squared values."

The code has

    ypred <- do_call(posterior_epred, args_ypred)
    ll <- do_call(log_lik, args_ypred)

I have a case where it makes sense to examine LOO-R^2 for posterior_epred(..., re_formula=NA). Now that re_formula=NA is passed also to log_lik() which makes the LOO part to be wrong, and the computed LOO-R^2 values are nonsense (e.g. -0.7, when the correct value is 0.08)

It seems some "further arguments" make sense to be passed to both posterior_epred() and log_lik(), like ndraws and draw_ids, so not passing any "further arguments" to log_lik() is not the solution.

@paul-buerkner
Copy link
Owner

I think we would have to add new list arguments args_epred and args_log_lik which would then only be passed to the individual functions. I will add this to my todo list for the next release. If you have time, you may of course already make a PR earlier.

@paul-buerkner paul-buerkner added this to the brms 2.23.0 milestone Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants