Collect coverage from doctests too

[FreezyLemon]

There are a few doctests that cover code which is not tested by separate unit tests.

I hope this change works as-is, the easiest way to test this change is to just open the PR and see what happens.

[FreezyLemon]

Oh, that uncovered a test failure.. I can't reproduce this locally. Any ideas?

[YeungOnion]

I also can't reproduce this, seems odd to fail there for precision when they're both on ubuntu-latest. I'll take a look.

---

Regarding the test failure,

Looks like this line fails,

test_case(0.3, 1, 0.2999999999999999888977697537484345957636833190918, pmf(1));

where the float is overspecified, playing with the IEEE-745 visualizer

0.29999999999999998... -> 0.3 // this is what's in the test case
0.29999999999999997    -> 0.3
0.29999999999999996    -> 0.29999999999999993

If we're going to specify a lot of precision, we should probably test_almost, but I'd also consider it to be a corner case to consider Binomial(p,1), so maybe that should be considered as well. But this would not explain why this fails when the coverage report is generated.

[YeungOnion]

It appears #261 that the nightly compiler causes similar effects in other tests as well.

 failures:
    distribution::binomial::tests::test_pmf
    distribution::fisher_snedecor::tests::test_ln_pdf
    distribution::fisher_snedecor::tests::test_pdf
    distribution::log_normal::tests::test_mean
    distribution::log_normal::tests::test_mode
    distribution::log_normal::tests::test_pdf
    distribution::weibull::tests::test_pdf
    distribution::weibull::tests::test_sf
    function::beta::tests::test_beta_reg
    function::exponential::tests::test_integral
    function::gamma::tests::test_inv_digamma

perhaps a lockfile would help here to see what changes across compile version.. I'll have to take a look later.

[YeungOnion]

All the test failures could be traced to precision differences in the functions module. I've not directly checked this by looking at values as the numerical methods iterate, but the distributions do rely on these functions which have failing cases.

ln mentions that it has unspecified precision since it calls an intrinsic. I know only enough to ask a question about this, but perhaps this is a motivator to look at different convergence or relying on 14 digits instead.

Perhaps if intrinsic fused multiply-add has regression testing for a larger number of digits then the precision might be better in our scenarios. However, I don't know enough about floating point to be confident if there are places we wouldn't obtain at least 15 digits.

EDITS:

1. fixed md link syntax for f64::ln docs

[FreezyLemon]

But this would not explain why this fails when the coverage report is generated.

Not sure, but I think the binary used for coverage gets compiled with some fairly specific compiler options. I think it's possible that there are cases where the same function/operation gets compiled into more floating-point operations than it would on a normal --release build, which could explain the decreased precision.

Or it could just be differences from one compiler version to another, I'm not sure. I would not worry too much about it unless the tests fail on normal runs.

This is still annoying though. Not sure it's worth the hassle fixing this, the coverage gained by doctests is probably not a lot. Thoughts?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.15%. Comparing base (44ccdb9) to head (4f42f27).
Report is 19 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #259      +/-   ##
==========================================
+ Coverage   89.49%   93.15%   +3.66%     
==========================================
  Files          50       52       +2     
  Lines       10851    11531     +680     
==========================================
+ Hits         9711    10742    +1031     
+ Misses       1140      789     -351     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[YeungOnion]

The fix/revert for rust-lang/rust#128386 went in, so this should have the same behavior as before.

I notice on my local machine that before updating nightly, binomial::tests had some failures. After a rustup update nightly it passed, so I'm confident we're good here.

Thanks for adding these!