[Insight-developers] Test framework

[Steve M. Robbins]

Hello,

On Wed, Dec 10, 2008 at 03:51:56PM -0500, Bill Lorensen wrote:

> Given the success of "Adopt a Bug", maybe we should initiate a "Cover a Class".

Now that I have some holiday time, I'd like to raise the issue of unit
testing frameworks.  

The few test codes that I've looked at are all written out long-hand,
without benefit of a modern unit testing framework such as CppUnit or
Boost.Test.  This makes writing a test case far more tedious than it
should be and, consequently, I have never bothered (I may have sent
/one/ to VTK).  If you truly want others to "Cover a Class", I believe
you need to lower the barrier to writing a test case.

Would the ITK developers consider the use of Boost.Test [1]?

I'm well aware that you are justifiably reticent to add extra
dependencies to impede the use of ITK.  I don't recall any list
traffic about using Boost in ITK, but Boost is pretty well established
now, portable, and is shipped with the major linux distributions.  I
would argue that you could consider Boost as an external
pre-requisite, akin to the requirement of a C++ compiler.  Moreover,
if only Boost.Test is used, suitable CMake magic can be used to ensure
the end-user doesn't need it but only those developers writing or
running Boost.Test-based tests.

Just to be clear: I'm not suggesting that the existing tests be
rewritten.  Nor am I advocating that Boost.Test be used for *all* new
tests.  I'm only suggesting that Boost.Test be allowed for those of us
who want to contribute new test cases.

So what is the advantage of Boost.Test for unit tests?  Clarity --
which is key for test code for two reasons.  First, you want the test
statements to stand out so you can see at a glance the logic of the
tests.  Second, non-test boilerplate code brings with it an increased
likelihood of bugs and you really don't want to be debugging the test
code!

As a small example of this, consider the following snippet from
Testing/Code/Common/itkImageRegionTest.cxx:

  try
    {
    SliceRegionType sliceA;
    sliceA = regionA.Slice(2);
    std::cout << "regionA.Slice(2): " << sliceA;
    }
  catch (itk::ExceptionObject &err)
    {
    std::cout << "Caught unexpected exception" << err;
    return EXIT_FAILURE;
    }

  try
    {
    SliceRegionType sliceA;
    sliceA = regionA.Slice(20);
    std::cout << "regionA.Slice(20): " << sliceA;
    std::cout << "Failed to catch expected exception" << std::endl;
    return EXIT_FAILURE;
    }
  catch (itk::ExceptionObject &err)
    {
    std::cout << "Caught expected exception" << err;
    }

All of that can be replaced by:

    BOOST_CHECK_EQUAL( slice2.mRegion, volume.mRegion.Slice( 2 ) );
    BOOST_CHECK_THROW( volume.mRegion.Slice( 20 ), std::exception );

In fact, this is not a direct replacement, but a distinct improvement:
the first line checks the return value of the Slice() call!

At the risk of belabouring this, I'll include a Boost.Test replacement
for the entire itkImageRegionTest.cxx.

Comments?

Best Regards,
-Steve

[1] http://www.boost.org/doc/libs/1_37_0/libs/test/doc/html/index.html



--------------------- itkImageRegionTest.cxx ---------------------------------

#define BOOST_AUTO_TEST_MAIN
#include <boost/test/auto_unit_test.hpp>

#include "itkImageRegion.h"


template< unsigned int VImageDimension >
struct Fixture
{
    typedef itk::ImageRegion<VImageDimension>            RegionType;
    typedef typename RegionType::IndexType               IndexType;
    typedef typename RegionType::SizeType                SizeType;

    RegionType mRegion;
};

struct Fixture1 : public Fixture<1>
{
    Fixture1( int start0,
	      int size0 )
    {
	IndexType start = {{ start0 }};
	SizeType  size  = {{ size0 }};
	mRegion = RegionType( start, size );
    }
};

struct Fixture2 : public Fixture<2>
{
    Fixture2( int start0, int start1,
	      int size0,  int size1 )
    {
	IndexType start = {{ start0, start1 }};
	SizeType  size  = {{ size0,  size1 }};
	mRegion = RegionType( start, size );
    }
};

struct Fixture3 : public Fixture<3>
{
    Fixture3( int start0, int start1, int start2,
	      int size0,  int size1,  int size2 )
    {
	IndexType start = {{ start0, start1, start2 }};
	SizeType  size  = {{ size0,  size1,  size2 }};
	mRegion = RegionType( start, size );
    }
};


BOOST_AUTO_TEST_CASE( testSlice )
{
    Fixture3 volume( 12, 12, 12, 10, 20, 30 );
    Fixture2 slice0( 12, 12, 20, 30 );
    Fixture2 slice1( 12, 12, 10, 30 );
    Fixture2 slice2( 12, 12, 10, 20 );

    BOOST_CHECK_EQUAL( slice0.mRegion, volume.mRegion.Slice( 0 ) );
    BOOST_CHECK_EQUAL( slice1.mRegion, volume.mRegion.Slice( 1 ) );
    BOOST_CHECK_EQUAL( slice2.mRegion, volume.mRegion.Slice( 2 ) );
}
    
BOOST_AUTO_TEST_CASE( testSliceOutOfBounds )
{
    Fixture3 volume( 12, 12, 12, 10, 20, 30 );

    BOOST_CHECK_THROW( volume.mRegion.Slice( -1 ), std::exception );
    BOOST_CHECK_THROW( volume.mRegion.Slice( 3 ), std::exception );
}

BOOST_AUTO_TEST_CASE( testVolumeIsInside )
{
    Fixture3 volumeA( 12, 12, 12, 10, 20, 30 );
    Fixture3 volumeB( 14, 14, 14,  5, 10, 15 );

    BOOST_CHECK(   volumeA.mRegion.IsInside( volumeB.mRegion ) );
    BOOST_CHECK( ! volumeB.mRegion.IsInside( volumeA.mRegion ) );
}

--------------------- itkImageRegionTest.cxx ---------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20081228/02903e0b/attachment.pgp>