Cannot produce differential coverage report showing methods in C++ header that are uncovered

[rael007]

Before I dive in, thanks for all the work on this project, we are finding the tools here very useful.

I have struggled for nearly half a day with this, and I may be making a glaring mistake, so I am raising the issue here rather than creating a bug report. First, before I lay out in detail what I'm doing, I will give a very brief overview in form of what I am doing, and will pose a general question. If the answer to my question is a resounding "yes, you should be able to do that", then I'll post more detail.

I am finding that no matter what I do, if I have baseline code committed and am able to build, run tests, etc., then some time later if I add a method to a class in my header file but I do not add a test for this new function in my test code, not only does the uncovered method in the header not get reported (after build, test, report generation), the header file itself does not show up in the report. If I then add a new fake test method to the class in the header and invoke that from within my test, the header file does show up in the coverage, and the new fake test method shows as being covered, but the previous uncovered method shows no indication that it is uncovered.

The general question is, then, using the latest lcov code is it possible to generate a differential coverage report that shows UNC (uncovered new code) for code that is in a C++ header file?

[henry2cox]

Short answer: yes, this works as you would expect it to.

Longer answer: only if the C++ code in question is instantiated. In particular: template functions or template methods which are not instantiated, do not have code generated for them - so they are simply not in the coverage data.
Similarly, inline functions which are not used will usually not appear - for the same reason. (Usually is moderately hard to define, as it depends on whether the compiler decides to out-line it, and a few other things. This is also version dependent - and likely, command-line dependent.)
It doesn't matter if the templates/inlines are in a header or local to some source file ; the same considerations apply.

[henry2cox]

Forgot to mention:

• Imagine the case that your new function/method is not instantiated (and is thus "not code") at the moment
• time passes - and you make a bunch of other changes, and eventually make a new release X.Y - and that code is still not instantiated.
  X.Y is your 'baseline'.
• now somebody changes something or adds a test - and that (now rather old) code is instantiated...
  So now it appears in your 'current' coverage data.

Source for that method is unchanged from your baseline, but now that source "is code" (because it appears in the coverage data).

• if your regression suite exercises the code: then it will fall into category GIC (not code before, is code and is hit now).
• if your regression suite doesn't hit the code: then it will fall into category UIC (not code before, is code now but is not hit).

This is the reason that the default differential coverage criteria script uses "LBC + UNC + UIC == 0" (see criteria.pm in the scripts directory)

[henry2cox]

Forgot to mention that the above also implies that your Jenkins hate-mailer should send its 'coverage criteria failed' automail to both the author of the most recent change (which caused this 'text-is-now-code' to appear in the report) as well as the author(s) of the text region itself. Either, both, or neither could be at fault...but this is a pretty good initial guess - and reduces the management load on the project lead.

[rael007]

Thanks for the reply Henry. The methods are not explicitly tagged as 'inline', nor are they templated. Here is a very streamlined clip of the code in question:

struct data_record_store_t {
// much elided here
      size_t size()
      {
          std::shared_lock l( m_data_record_store );
          return data_record_store.size();
      }

      size_t storage()
      {
          std::shared_lock l( m_data_record_store );
          size_t           s { 0 };
          for ( const auto& [ k, data_record ] : data_record_store )
          {
              s += data_record.data.size();
          }
          return s;
      }
};

I am finding that the lcov invocation is simply not producing information for this header file, or rather, that the information section in the .info files are basically empty. Here is the relevant section from the baseline .info file (directory structure elided with [...]) in the SF line below):

TN:
SF:[...]/dev/kudos/data_record_store.hh
VER:SHA 1426e9f4eedb8baa97d73aa76471d1fd7732bfc4
FNF:0
FNH:0
LF:0
LH:0
end_of_record

and here it is from the current .info file:

TN:
SF:[...]/dev/kudos/data_record_store.hh
VER:SHA a3375a81cb789d8bfb9953b0dd267e7a16c1a6ab
FNF:0
FNH:0
LF:0
LH:0
end_of_record

The invocation of lcov follows the lcov example, though I added the '-all' in an effort to get my header code to show up, is basically the following (I have the latest lcov code in $HOME/dev/lcov):

$HOME/dev/lcov/bin/lcov -all --branch --gcov-tool /usr/bin/gcov-13 \
    --directory . --exclude "t_*" --exclude '/usr/*' \
    --capture --output-file kudos-baseline.info \
    --version-script $HOME/dev/lcov/scripts/gitversion.pm

[henry2cox]

class (struct) methods whose definition are in the body of the class are inline (see language spec) - so the coverage artifacts are behaving more-or-less as one might expect (especially if your actual data_record_store_t is a template class).

[rael007]

Ok, I was wondering about this, makes complete sense. Thanks very much for the explanation.