improve Pareto k warning message

[jgabry]

Submission Checklist

• Run unit tests: ./runTests.py src/test/unit
• Run cpplint: make cpplint
• Declare copyright holder and open-source license: see below

Summary

Fixes #3223. Improves misleading Pareto k warning message. I didn't see any tests that check the text of this message so I didn't edit any tests.

The old message said "...which often indicates model misspecification", but this is somewhat misleading. More accurate is
"... which may indicate a poor approximation of the target distribution or model misspecification".

@avehtari What do you think of this message?

Intended Effect

Change text of warning message.

How to Verify

I suppose run a model that results in the warning message (are there any we know to definitely trigger the warning? Aki will probably know), but I think it’s sufficient to just look at the edited text.

Side Effects

Documentation

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

• Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
• Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[jgabry]

I asked @avehtari for feedback but if anyone else has ideas for the content of the message that would be great too. Ideally in the next couple of days so this can be merged before the release.

[avehtari]

The model misspecification is not relevant here, and probably was mistakenly carried over from LOO-CV diagnostic.

I would write
"Pareto k diagnostic value is greater than 0.7 which may indicate a poor approximation and indicates that importance resampling is not able to improve the approximation."

[jgabry]

"Pareto k diagnostic value is greater than 0.7 which may indicate a poor approximation and indicates that importance resampling is not able to improve the approximation."

Does this slightly edited version convey the right thing to the user?

“The Pareto k diagnostic value is greater than 0.7. Importance resampling was not able to improve the approximation, which may indicate that the approximation itself is poor.”

[avehtari]

ok

[jgabry]

Ok, now updated

[jgabry]

Will go ahead and merge since approved and everything is passing.