CTest/Coverage

Coverage With C++

Currently coverage is only supported on gcc compiler. To perform coverage test, make sure that your code is build with debug symbols, without optimization, and with special flags. These flags are:

-fprofile-arcs -ftest-coverage

Also make sure to pass these flags to C compiler, CXX compiler, and the linker. For example:

CXXFLAGS="-g -O0 -Wall -W -Wshadow -Wunused-variable \
 -Wunused-parameter -Wunused-function -Wunused -Wno-system-headers \
 -Wno-deprecated -Woverloaded-virtual -Wwrite-strings -fprofile-arcs -ftest-coverage"
CFLAGS="-g -O0 -Wall -W -fprofile-arcs -ftest-coverage"
LDFLAGS="-fprofile-arcs -ftest-coverage"

Shared libs

You might have issues with shared libs, as reported here:

• https://twiki.cern.ch/twiki/bin/view/SPI/HowToTestCoverageReports

(I was not able to use that trick, on a debian/oldstable and gcc 3.3.5, ref here).

Other refs found on the net:

• http://lists.gforge.inria.fr/pipermail/visp-commits/2006-November/000273.html
• http://www.spinics.net/lists/gcchelp/msg06666.html

Conclusion: You should use gcc 3.4 an up

Coverage With Python

Simple case

$ cat hello.py

"""
Comment #1
"""
def untested():
  print "untested"

"""
Comment #2
"""
def tested():
  print "tested"

"""
Comment #3
"""
if __name__ == '__main__':
  tested()

$ python /usr/lib/python2.3/trace.py -c --coverdir=. --ignore-dir /usr/lib/python2.3 hello.py

will produce a file

hello.cover

$ cat hello.cover

       """
       Comment #1
    1: """
    1: def untested():
         print "untested"

       """
       Comment #2
       """
    1: def tested():
    1:   print "tested"

       """
       Comment #3
       """
    1: if __name__ == '__main__':
    1:   tested()

Complex case

Using python code from VTK:

VTK/Wrapping/Python/vtk/test/Testing.py

$ python /usr/lib/python2.3/trace.py -c --coverdir=. --ignore-dir /usr/lib/python2.3 Testing.py

will produce:

ls *.cover
Testing.cover       vtk.__init__.cover  vtk.filtering.cover         vtk.graphics.cover  vtk.imaging.cover  vtk.io.cover        vtk.rendering.cover      vtk.util.__init__.cover      vtk.util.vtkMethodParser.cover  vtk.widgets.cover
vtk.__helper.cover  vtk.common.cover    vtk.genericfiltering.cover  vtk.hybrid.cover    vtk.infovis.cover  vtk.parallel.cover  vtk.test.BlackBox.cover  vtk.util.vtkConstants.cover  vtk.volumerendering.cover

Example:

• Testing.cover
• vtk.util.vtkMethodParser.cover