-
Notifications
You must be signed in to change notification settings - Fork 4
Recommendations for problematic diagnostics
There is no one size fits all causes and remedies so consider these as examples.
Consistency and independent posterior sample assumptions are required for rank uniformity in SBC. Consistency is related to coverage such as "In 95% of simulations, the true parameter lies within the central 95% posterior credible interval." 95% could be replaced by any ratio which is specified by uniformity
in rank histogram i.e. at any quantile q from 0 to 1, the ratio of smaller posterior samples compared to prior value is q. Breach of the two requirements is visually expressed in graphics. Independence could be improved without improving the model via increasing thin_rank
. Therefore upon meeting the plot 1 and 3, increasing thin_rank
is recommended. Assuming independence, further diagnosis on bias or dispersion could be given.
Tips for differentiating (1 vs 4) and (2 vs 3) from ecdf_diff
plot is whether out-of-boundary (OOB) sample ecdf quadrants are adjacent or not.
Without correlation, it is likely that adjacent OOB indicates a bias problem while nonadjacent implies variance problem.
The following are from Teemu's talk in SBC Stanconnect here.