[vtk-developers] VTK test code coverage

[Bill Lorensen]

Folks,

VTK's test code coverage is not very good. Currently it stands at
about 62% of the code. I think the pre/post modularization coverage
rates are similar.

One of the lessons we learned from ITK's modularization is that many
of the ITK tests were integration/regression tests versus unit
tests. Integration/regression tests tend to test multiple classes, but
fail to fully cover the code in individual classes.

Integration/regression tests are certainly important because they test
the code as it may be used in an application.  In fact, many of VTK's tests
are derived from examples in the original text. Their purpose was to make
sure the code actually did what the text said it would do.

But, having 40% of VTK's code untested is alarming.

Last year, I picked valgrind defects as my Fall Project:
http://itk.org/Wiki/VTK/SoftwareQuality/Valgrind/Fall2011

I have adapted some ITK scripts to easily evaluate code coverage for VTK.
There is a script in
Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh that
facilitates generating code coverage for a set of unit tests.


For example, running this command from your build directory
(instrumented for coverage)

../VTK/Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh
Domains/Chemistry -R vtkDomainsChemistry
shows that the Domains/Chemistry module has 67.6% code coverage
while
../VTK/Utilities/Maintenance/computeCodeCoverageLocallyForOneTest.sh
Geovis/Core -R vtkGeovisCore
shows that the Geovis/Core modules has 11.2% code coverage/

This year's Fall Project is code coverage. This is a bit ironic,
because for my GE Six Sigma Project in 1995, I chose code
coverage.

What can you do to help?
If you are adding or modifying new classes to vtk, please try to
add/enhance the tests for your class.
I'll be adding gerrit topics for tests this fall. Please help me push them
through the review process



Also, I'll be rejecting gerrit topics that add new classes without good
unit tests.

Bill
'



-- 
Unpaid intern in BillsBasement at noware dot com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20120904/25d404d3/attachment.html>