[Insight-developers] Test framework

[Daniel Blezek]

Hi Bill,

  I would add the Google Test framework to the mix.  We've been using it
extensively in our lab.  It meets 2, 3, 4, 5, 6, 7.  Not sure about platform
support across _all_ ITK's platform, but I've never had a compile warning
from it nor any memory leaks.  Given that they support several embedded
platforms, I'd bet it compiles cleanly on all our platforms.  It's a very
minimalist library, mostly implemented through macros and supports many of
the nice ctest features, i.e. run every N'th test, run all matching tests,
etc.

http://code.google.com/p/googletest/

I found a CMake macro that parses the source code and automatically adds
tests making it trivial to add new tests:

macro(ADD_GOOGLE_TESTS executable)
  foreach ( source ${ARGN} )
    file(READ "${source}" contents)
    string(REGEX MATCHALL "TEST_?F?\\(([A-Za-z_0-9 ,]+)\\)" found_tests
${contents})
    foreach(hit ${found_tests})
      string(REGEX REPLACE ".*\\(([A-Za-z_0-9]+)[, ]*([A-Za-z_0-9]+)\\).*"
"\\1.\\2" test_name ${hit})
      add_test(${test_name} ${executable} --gtest_filter=${test_name}
${MI3CTestingDir})
    endforeach(hit)
  endforeach()
endmacro()


# Add all tests found in the source code, calling the executable to run them
add_google_tests ( ${EXECUTABLE_OUTPUT_PATH}/NoOp ${mi3cTestSource})

The main looks like this:

#include <gtest/gtest.h>

int main(int argc, char* argv[])
{
  testing::InitGoogleTest ( &argc, argv );
  return RUN_ALL_TESTS();
}

and a test looks like this (we use MD5 hashes to test for pass/fail):

#include <iostream>

#include <mi3cImage.h>
#include <mi3cImageLoader.h>
#include <gtest/gtest.h>
#include <NoOp.h>

TEST(IO, LoadCT) {
  mi3c::ImageLoader loader;
  mi3c::Image::Pointer image = loader.setFilename ( dataFinder.getFile (
"CT.hdr" ) ).execute();
  ASSERT_EQ ( "c1d43aaa5b991431a9daa1dc4b55dbb1", image->getMD5() );
}


Fixtures are also supported to remove redundant code.

Cheers,
-dan

On Mon, Dec 29, 2008 at 10:35 AM, Bill Lorensen <bill.lorensen at gmail.com>wrote:

> I agree that we should look at a unit test facility.
>
> We should start a list of requirements. For example,
>
> 1) Must support all itk platforms.
> 2) We must be able to distribute it with itk.
> 3) It must fit within our test harness facility. Recall that we try to
> minimize the number of executables by combining large numbers of tests
> into FooTests.cxx files.
> 4) It must be compatible with cmake/ctest/cdash. For example, a test
> must be able to "return EXIT_SUCCESS" and "return EXIT_FAILURE".
> 5) It should not add complexity to an already complex testing process.
> 6) It must be compatible with itk's strict attention to compile
> warnings and dynamic memory anlysis. In other words, it must not
> produce warnings or purify defects.
> 7) It should have a minimal source footprint.
>
> These are just a few requirements off the top of my head. I'll add
> them to the new page when I get a chance,
>
> Bill
>
> On Mon, Dec 29, 2008 at 10:47 AM, Luis Ibanez <luis.ibanez at kitware.com>
> wrote:
> >
> > Hi Mathieu,
> >
> > Minor correction:
> >
> > The text of the UnitTestCpp license:
> >
> https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/UnitTest++/COPYING
> >
> > doesn't correspond to a BSD license:
> > http://www.opensource.org/licenses/bsd-license.php
> >
> > but to an MIT license:
> > http://www.opensource.org/licenses/mit-license.php
> >
> > which is a technicality after all, since both licenses could be combined
> > without any conflict.   :-)
> >
> >
> > -----
> >
> >
> > Could you tell us more about your experience with UnitTestCpp  ?
> >
> > or if you prefer, could you add your comments to the Wiki page:
> >
> http://www.itk.org/Wiki/Proposals:Increasing_ITK_Code_Coverage#UnitTestCpp
> >
> >
> > The size of the package looks fine. It could fit neatly in
> > Insight/Utilities...
> >
> >
> >  Thanks
> >
> >
> >     Luis
> >
> >
> > --------------------------
> > Mathieu Malaterre wrote:
> >>
> >> On Mon, Dec 29, 2008 at 5:29 AM, Steve M. Robbins <steve at sumost.ca>
> wrote:
> >>
> >>> Hello,
> >>>
> >>> On Wed, Dec 10, 2008 at 03:51:56PM -0500, Bill Lorensen wrote:
> >>>
> >>>
> >>>> Given the success of "Adopt a Bug", maybe we should initiate a "Cover
> a
> >>>> Class".
> >>>
> >>> Now that I have some holiday time, I'd like to raise the issue of unit
> >>> testing frameworks.
> >>>
> >>> The few test codes that I've looked at are all written out long-hand,
> >>> without benefit of a modern unit testing framework such as CppUnit or
> >>> Boost.Test.
> >>
> >>
> >> ... or UnitTestCpp :)
> >>
> >> http://unittest-cpp.sourceforge.net/
> >>
> >> Which comes with a BSD like license ;)
> >>
> >>
> >>
> https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/UnitTest++/COPYING
> >>
> >> I wrote a cmakelists.txt file for it:
> >>
> >> http://gdcm.svn.sf.net/viewvc/gdcm/Sandbox/UnitTest%2B%2B/
> >>
> >> src files are very lightweight "du -sk src" return 712
> >>
> >>
> >> 2cts
> >
> _______________________________________________
> Insight-developers mailing list
> Insight-developers at itk.org
> http://www.itk.org/mailman/listinfo/insight-developers
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20081229/606e3419/attachment.htm>