Add test coverage reporting

[rkm]

No description provided.

[tznind]

This issue is likely to be a barrier:

OpenCover/opencover#703

I think we would need a windows CI build to do it currently. Definetly worth doing though.

[rkm]

Is that the only option? 😞

Just had a very quick look, and seems like this may now be built-into dotnet: https://medium.com/swlh/generating-code-coverage-reports-in-dotnet-core-bb0c8cca66

Edit: Other option is https://github.com/tonerdo/coverlet

[tznind]

It's certainly doable https://coveralls.io/github/tznind/StarshipWanderer?branch=master but it's a bit of a pain and for me the easiest way seemed to be building under windows (with appveyor in this case).

This was the command (along with some package references and a NuGet.Config to make packages actually download to the repo/pacakges directory - instead of user profile!)

 - cmd: "\"packages/opencover/4.7.922/tools/OpenCover.Console.exe\" -target:\"c:/program files/dotnet/dotnet.exe\" -targetargs:\"test -f netcoreapp3.1 -c Release /p:DebugType=full\" -filter:\"+[*Wanderer*]*\"  -output:coverage.xml -register:appveyor -oldStyle -hideskipped:File"

[rkm]

Just tried the following:

$ dotnet add package coverlet.collector
$ dotnet test --collect:"XPlat Code Coverage"
$ pip install pycobertura
$ pycobertura show TestResults\f3d23c47-7901-4264-9cc6-b80dd7a27dd2\coverage.cobertura.xml
Filename                                                   Stmts    Miss  Cover    Missing
-------------------------------------------------------  -------  ------  -------  -------------------------------------------------------------------------
RabbitMQAdapter.cs                                           249     249  0.00%    27-442, 44-88
TimeTracker.cs                                                 8       8  0.00%    29-31, 19-23
Options\CliOptions.cs                                          2       2  0.00%    15-23
Options\ConsumerOptions.cs                                    13       8  38.46%   23, 36-42
Options\GlobalOptions.cs                                      54      23  57.40%   34, 38, 50-55, 76, 84, 88, 91-111
Options\ProducerOptions.cs                                    28      24  14.28%   17, 30-66
Messaging\BatchProducerModel.cs                                4       4  0.00%    27-29, 15
Messaging\Consumer.cs                                         99      54  45.45%   21-26, 65, 75, 80, 91, 96-103, 111, 131-185, 198, 209-211, 218-220, 48-52
Messaging\ControlMessageConsumer.cs                          114     114  0.00%    66-232, 19-56
Messaging\ProducerModel.cs                                    57      57  0.00%    78-150, 32-67
Messages\AccessionDirectoryMessage.cs                         29      16  44.82%   37-39, 50-51, 64-71, 29-35
Messages\DicomFileMessage.cs                                  79      54  31.64%   26-44, 67-114, 132-133, 152-164, 56-64
Messages\FatalErrorMessage.cs                                 20      20  0.00%    10-43, 16-20
Messages\MessageHeader.cs                                    111      38  65.76%   107-109, 126-134, 156-164, 169-170, 172, 190-197, 83-90
Messages\RabbitMqXDeathHeaders.cs                             69      69  0.00%    12-123, 33-63
Messages\SeriesMessage.cs                                     36      12  66.66%   23-41, 65-66, 85-92
Messages\TagPromotionMessage.cs                               23      23  0.00%    13-60
Messages\Extraction\ExtractFileCollectionInfoMessage.cs       30      20  33.33%   38-57, 61-62, 64, 68-76, 44-47
Messages\Extraction\ExtractFileMessage.cs                     24      24  0.00%    16-62, 27-30
Messages\Extraction\ExtractFileStatusMessage.cs               34      34  0.00%    15-82, 37
Messages\Extraction\ExtractionRequestInfoMessage.cs           29      29  0.00%    10-59, 17
Messages\Extraction\ExtractionRequestMessage.cs               29      16  44.82%   38-40, 46-47, 53-54, 56, 60-69
Messages\Extraction\ExtractMessage.cs                         42      24  42.85%   45-46, 55-82, 38-39
Messages\Extraction\IsIdentifiableMessage.cs                  37      37  0.00%    8-73, 23-44
MessageSerialization\JsonCompatibleDictionary.cs              21       1  95.23%   54
MessageSerialization\JsonConvert.cs                           28      15  46.42%   42, 47-52, 62-70, 21-24
Helpers\MicroserviceObjectFactory.cs                          29      29  0.00%    28-76, 11
Execution\MicroserviceHost.cs                                 97      97  0.00%    127-215, 113-119, 30-120
Execution\MicroserviceHostBootstrapper.cs                     33      33  0.00%    24-67, 18-21
Events\FatalErrorEventArgs.cs                                 12       7  41.66%   9-10, 19-23
TOTAL                                                       1440    1141  19.01%

[tznind]

That's cool, is that file format supported by Coveralls / codecov.io ? If so happy for you to integrate that into travis.

[rkm]

Not sure, but i think it outputs a standard xml format.

I've just been pointed to this new tool as well: https://github.com/microsoft/ApplicationInspector. Seems interesting but not had a chance to poke about yet.

[tznind]

Looks a bit like lgtm and snyk. I'm all for any tool that runs from CI and outputs to a sensible webhost (i.e. doesn't require looking at build logs and has a notification system for PR / failures etc). Lets descope this till we have MVP up and running. Integration repo will give us good confidence in new features but this would be extra icing.