<div dir="ltr"><div><div><div><div><div>Hi,<br></div>I'd definitely would give it a try. <br><br></div>I'm aware that getting coverage results may delay the merge process, but depending on the delay I think it is worthwhile.<br><br></div><div>I had a look at the --diff flag, but it did not provide a clear picture of how to proceed.<br><br></div>As for the results, an option would be to present the coverage figure along with a link to a kind of dashboard showing the files (and the lines in these files; much like the current file-wise diff presentation) whose coverage has changed with respect to the master/nightly. A separate message from the dashboard builds sent to the review would be a solution, rather than holding back the dashboard build results message until both tasks are completed.<br><br></div>It may take us a while, but I think it adds a value to the toolkit development process.<br><br></div>We may start narrowing down the space of solutions and hopefully have a first version of the tool and do some tests to see how it behaves in a few weeks.<br><div><div><div><div><div><div><div><div><div class="gmail_extra"><br></div><div class="gmail_extra">JON HAITZ<br><br clear="all"></div><div class="gmail_extra"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><table style="color:rgb(0,0,0);font-family:"times new roman";font-size:medium" cellspacing="0" cellpadding="0" border="0" align="center"><tbody><tr><td></td></tr></tbody></table><pre style="color:rgb(0,0,0);white-space:pre-wrap"><br></pre><div><div dir="ltr"></div></div></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On 17 January 2017 at 19:28, Matt McCormick <span dir="ltr"><<a href="mailto:matt.mccormick@kitware.com" target="_blank">matt.mccormick@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Jon,<div><br></div><div>This is a great idea.</div><div><br></div><div>We can create coverage metrics with gcov / lcov as in this script:</div><div><br></div><div> <a href="https://github.com/InsightSoftwareConsortium/ITK/blob/d125b69c44ac0ed7d61df0252a6b3b054c60df83/Utilities/Maintenance/computeCodeCoverageLocally.sh" target="_blank">https://github.com/<wbr>InsightSoftwareConsortium/ITK/<wbr>blob/<wbr>d125b69c44ac0ed7d61df0252a6b3b<wbr>054c60df83/Utilities/<wbr>Maintenance/<wbr>computeCodeCoverageLocally.sh</a></div><div><br></div><div>and it looks like lcov has a --diff flag:</div><div><br></div><div> <a href="http://ltp.sourceforge.net/coverage/lcov/lcov.1.php" target="_blank">http://ltp.sourceforge.net/<wbr>coverage/lcov/lcov.1.php</a></div><div><br></div><div>A new script (computeCodeCoverageForPatch.<wbr>sh?) could be created. </div><div><br></div><div>We could run this either with every patch or on demand if it is resource intensive. It may take more effort to figure out a way to post the HTML output and create a link, but it should be doable.</div><div><br></div><div>Thanks,</div><div>Matt </div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-h5">On Tue, Jan 17, 2017 at 11:58 AM, Jon Haitz Legarreta <span dir="ltr"><<a href="mailto:jhlegarreta@vicomtech.org" target="_blank">jhlegarreta@vicomtech.org</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-h5"><div dir="ltr"><div><div><div><div>Hi,<br></div>I'm wondering whether with the current code coverage tools/CI used, and without excessive work and burden, we would be able to spot merge requests or commits that lower the code coverage.<br><br></div>That is, automatically performing a kind of diff between two code coverage reports and identifying the commits (and, thus, the files) that actually make the coverage figure change. Or at least, those that make the figure decrease.<br><br></div><div>If I'm not mistaken some CI tools already provide a coverage report (or at least the figure) for each commit (not 100% sure). I guess that would ideally allow maintainers to take action before a topic or commit makes its way into master.<br><br>For example, that would allow to automatically ask the developer to raise the coverage, or setting some policy to require at least say, a 95% coverage in new files, or to keep the coverage figure when modifying existing files, although this last may be unfeasible in practice. <br><br></div><div>I am well aware that even if this were possible, it would require some investigation. I am ready to help.<br><br></div>Thanks,<br></div>JON HAITZ<br><br clear="all"><div><div><div><div><div><div><div><div class="gmail-m_8045704052314287462m_-4465183733691499395gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><table style="color:rgb(0,0,0);font-family:"times new roman";font-size:medium" cellspacing="0" cellpadding="0" border="0" align="center"><tbody><tr><td></td></tr></tbody></table><pre style="color:rgb(0,0,0);white-space:pre-wrap">--<br><br></pre><div><div dir="ltr"></div></div></div></div></div></div></div></div></div></div></div></div>
</div></div></div></div></div></div></div>
<br></div></div>______________________________<wbr>_________________<br>
Community mailing list<br>
<a href="mailto:Community@itk.org" target="_blank">Community@itk.org</a><br>
<a href="http://public.kitware.com/mailman/listinfo/community" rel="noreferrer" target="_blank">http://public.kitware.com/mail<wbr>man/listinfo/community</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div></div></div></div></div></div></div></div></div></div>