[ITK] A suggestion to manage code coverage drops

[Matt McCormick]

Hi Jon,

This is a great idea.

We can create coverage metrics with gcov / lcov as in this script:


https://github.com/InsightSoftwareConsortium/ITK/blob/d125b69c44ac0ed7d61df0252a6b3b054c60df83/Utilities/Maintenance/computeCodeCoverageLocally.sh

and it looks like lcov has a --diff flag:

  http://ltp.sourceforge.net/coverage/lcov/lcov.1.php

A new script (computeCodeCoverageForPatch.sh?) could be created.

We could run this either with every patch or on demand if it is resource
intensive. It may take more effort to figure out a way to post the HTML
output and create a link, but it should be doable.

Thanks,
Matt

On Tue, Jan 17, 2017 at 11:58 AM, Jon Haitz Legarreta <
jhlegarreta at vicomtech.org> wrote:

> Hi,
> I'm wondering whether with the current code coverage tools/CI used, and
> without excessive work and burden, we would be able to spot merge requests
> or commits that lower the code coverage.
>
> That is, automatically performing a kind of diff between two code coverage
> reports and identifying the commits (and, thus, the files) that actually
> make the coverage figure change. Or at least, those that make the figure
> decrease.
>
> If I'm not mistaken some CI tools already provide a coverage report (or at
> least the figure) for each commit (not 100% sure). I guess that would
> ideally allow maintainers to take action before a topic or commit makes its
> way into master.
>
> For example, that would allow to automatically ask the developer to raise
> the coverage, or setting some policy to require at least say, a 95%
> coverage in new files, or to keep the coverage figure when modifying
> existing files, although this last may be unfeasible in practice.
>
> I am well aware that even if this were possible, it would require some
> investigation. I am ready to help.
>
> Thanks,
> JON HAITZ
>
> --
>
>
> _______________________________________________
> Community mailing list
> Community at itk.org
> http://public.kitware.com/mailman/listinfo/community
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/community/attachments/20170117/c093798a/attachment.html>