Proposals:Increasing ITK Code Coverage

Motivation

ITK currently has a 80.4% code coverage.

This means that about 30,000 lines of code are not tested.

We could significantly increase the code coverage of the toolkit, and in the process reduce the number of hidden bugs, by asking volunteers to adopt particular classes and write additional tests for increasing their code coverage. This could be done at the image of the "Adopt a Bug" program.

Infrastructure

It has been pointed out that the current testing infrastructure of ITK impose a high threshold of effort on contributors of new tests.

There are some existing unit test harnesses that might decrease the effort and provide additional functionality. A unit testing package for ITK must meet the following requirements:

1. It must have an itk-compatible license.
2. We must be able to distribute it with itk.
3. It must support all itk platforms.
4. It must fit within the itk test harness facility. Recall that we try to minimize the number of executables by combining large numbers of tests into FooTests.cxx files.
5. It must be compatible with cmake/ctest/cdash. For example, a test must be able to "return EXIT_SUCCESS" and "return EXIT_FAILURE".
6. It must not add complexity to an already complex testing process.
7. It must be compatible with itk's strict attention to compile warnings and dynamic memory anlysis. In other words, it must not produce warnings or purify defects.
8. It should have a minimal source footprint.

Suggestions for improving the testing system to make easier for contributors to introduce new tests include

Boost Test

• http://www.boost.org/doc/libs/1_37_0/libs/test/doc/html/index.html

Suggested by Steve Robbins

• http://www.itk.org/mailman/private/insight-developers/2008-December/011421.html

How it could work

--------------------- itkImageRegionTest.cxx ---------------------------------

#define BOOST_AUTO_TEST_MAIN
#include <boost/test/auto_unit_test.hpp>

#include "itkImageRegion.h"


template< unsigned int VImageDimension >
struct Fixture
{
    typedef itk::ImageRegion<VImageDimension>            RegionType;
    typedef typename RegionType::IndexType               IndexType;
    typedef typename RegionType::SizeType                SizeType;

    RegionType mRegion;
};

struct Fixture1 : public Fixture<1>
{
    Fixture1( int start0,
	      int size0 )
    {
	IndexType start = {{ start0 }};
	SizeType  size  = {{ size0 }};
	mRegion = RegionType( start, size );
    }
};

struct Fixture2 : public Fixture<2>
{
    Fixture2( int start0, int start1,
	      int size0,  int size1 )
    {
	IndexType start = {{ start0, start1 }};
	SizeType  size  = {{ size0,  size1 }};
	mRegion = RegionType( start, size );
    }
};

struct Fixture3 : public Fixture<3>
{
    Fixture3( int start0, int start1, int start2,
	      int size0,  int size1,  int size2 )
    {
	IndexType start = {{ start0, start1, start2 }};
	SizeType  size  = {{ size0,  size1,  size2 }};
	mRegion = RegionType( start, size );
    }
};


BOOST_AUTO_TEST_CASE( testSlice )
{
    Fixture3 volume( 12, 12, 12, 10, 20, 30 );
    Fixture2 slice0( 12, 12, 20, 30 );
    Fixture2 slice1( 12, 12, 10, 30 );
    Fixture2 slice2( 12, 12, 10, 20 );

    BOOST_CHECK_EQUAL( slice0.mRegion, volume.mRegion.Slice( 0 ) );
    BOOST_CHECK_EQUAL( slice1.mRegion, volume.mRegion.Slice( 1 ) );
    BOOST_CHECK_EQUAL( slice2.mRegion, volume.mRegion.Slice( 2 ) );
}
    
BOOST_AUTO_TEST_CASE( testSliceOutOfBounds )
{
    Fixture3 volume( 12, 12, 12, 10, 20, 30 );

    BOOST_CHECK_THROW( volume.mRegion.Slice( -1 ), std::exception );
    BOOST_CHECK_THROW( volume.mRegion.Slice( 3 ), std::exception );
}

BOOST_AUTO_TEST_CASE( testVolumeIsInside )
{
    Fixture3 volumeA( 12, 12, 12, 10, 20, 30 );
    Fixture3 volumeB( 14, 14, 14,  5, 10, 15 );

    BOOST_CHECK(   volumeA.mRegion.IsInside( volumeB.mRegion ) );
    BOOST_CHECK( ! volumeB.mRegion.IsInside( volumeA.mRegion ) );
}

--------------------- itkImageRegionTest.cxx ---------------------------------

UnitTestCpp

• http://unittest-cpp.sourceforge.net/

Suggested by Mathieu Malaterre

• http://www.itk.org/mailman/private/insight-developers/2008-December/011423.html

This package is distributed under an MIT License:

• https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/UnitTest++/COPYING