Skip to content

Recommendations for problematic diagnostics

Hyunji Moon edited this page Jul 3, 2022 · 4 revisions

There is no one size fits all causes and remedies so consider these as examples.

Consistency and independent posterior sample assumptions are required for rank uniformity in SBC. Consistency is related to coverage such as "In 95% of simulations, the true parameter lies within the central 95% posterior credible interval." 95% could be replaced by any ratio which is specified by uniformity in rank histogram i.e. at any quantile q from 0 to 1, the ratio of smaller posterior samples compared to prior value is q. Breach of the two requirements is visually expressed in graphics. Independence could be improved without improving the model via increasing thin_rank. Therefore upon meeting the plot 1 and 3, increasing thin_rank is recommended. Assuming independence, further diagnosis on bias or dispersion could be given.

Tips for differentiating (1 vs 4) and (2 vs 3) from ecdf_diff plot is whether out-of-boundary (OOB) sample ecdf quadrants are adjacent or not. 
Without correlation, it is likely that adjacent OOB indicates a bias problem while nonadjacent implies variance problem.

The following are from Teemu's talk in SBC Stanconnect here.

image
Clone this wiki locally