[vtk-developers] VTK test code coverage

Yuanxin Liu leo.liu at kitware.com
Wed Sep 5 14:46:16 EDT 2012


Hi, Bill,
  A separate but related question:  What should be the default behavior of
tests regarding "ERROR" strings in the standard output?  Currently, unless
you explicitly do something like set_tests_properties(.... PROPERTIES
FAIL_REGULAR_EXPRESSION "Error") the tests will pass.  This seems a little
dangerous to me.

Leo

On Tue, Sep 4, 2012 at 2:58 PM, Bill Lorensen <bill.lorensen at gmail.com>wrote:

> Folks,
>
> VTK's test code coverage is not very good. Currently it stands at
> about 62% of the code. I think the pre/post modularization coverage
> rates are similar.
>
> One of the lessons we learned from ITK's modularization is that many
> of the ITK tests were integration/regression tests versus unit
> tests. Integration/regression tests tend to test multiple classes, but
> fail to fully cover the code in individual classes.
>
> Integration/regression tests are certainly important because they test
> the code as it may be used in an application.  In fact, many of VTK's
> tests are derived from examples in the original text. Their purpose was to
> make sure the code actually did what the text said it would do.
>
> But, having 40% of VTK's code untested is alarming.
>
> Last year, I picked valgrind defects as my Fall Project:
> http://itk.org/Wiki/VTK/SoftwareQuality/Valgrind/Fall2011
>
> I have adapted some ITK scripts to easily evaluate code coverage for VTK.
> There is a script in
> Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh that
> facilitates generating code coverage for a set of unit tests.
>
>
> For example, running this command from your build directory
> (instrumented for coverage)
>
> ../VTK/Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh
> Domains/Chemistry -R vtkDomainsChemistry
> shows that the Domains/Chemistry module has 67.6% code coverage
> while
> ../VTK/Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh
> Geovis/Core -R vtkGeovisCore
> shows that the Geovis/Core modules has 11.2% code coverage/
>
> This year's Fall Project is code coverage. This is a bit ironic,
> because for my GE Six Sigma Project in 1995, I chose code
> coverage.
>
> What can you do to help?
> If you are adding or modifying new classes to vtk, please try to
> add/enhance the tests for your class.
> I'll be adding gerrit topics for tests this fall. Please help me push them
> through the review process
>
>
>
> Also, I'll be rejecting gerrit topics that add new classes without good
> unit tests.
>
> Bill
> '
>
>
>
> --
> Unpaid intern in BillsBasement at noware dot com
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20120905/fc4d9705/attachment.html>


More information about the vtk-developers mailing list