[Insight-developers] Test framework

Mon Dec 29 16:12:27 EST 2008

Yep, it is cool.  We've been using it extensively and I really like it.
On reflection, I'm not sure it will be a drop in replacement for the
test routines currently in use, without some messing around with the
internals.  The framework manages test discovery and execution and
leaves it intentionally opaque.  However, a good strategy might be to
leave the existing tests in place and start using a new test framework,
i.e., make itkCommonTests3 use the new framework.

I documented our experiences with Google Test on the wiki as you
suggested:
http://www.itk.org/Wiki/Proposals:Increasing_ITK_Code_Coverage#Google_Te
st

Cheers,
-dan

-----Original Message-----
From: insight-developers-bounces at itk.org
[mailto:insight-developers-bounces at itk.org] On Behalf Of Bill Lorensen
Sent: Monday, December 29, 2008 12:17 PM
To: Daniel Blezek
Cc: insight-developers at itk.org
Subject: Re: [Insight-developers] Test framework

Dan,

Sounds cool. Can you add some info to:

http://www.itk.org/Wiki/Proposals:Increasing_ITK_Code_Coverage

Bill

On Mon, Dec 29, 2008 at 1:14 PM, Daniel Blezek <daniel.blezek at gmail.com>
wrote:
> Hi Bill,
>
>   I would add the Google Test framework to the mix.  We've been using 
> it extensively in our lab.  It meets 2, 3, 4, 5, 6, 7.  Not sure about

> platform support across _all_ ITK's platform, but I've never had a 
> compile warning from it nor any memory leaks.  Given that they support

> several embedded platforms, I'd bet it compiles cleanly on all our 
> platforms.  It's a very minimalist library, mostly implemented through

> macros and supports many of the nice ctest features, i.e. run every 
> N'th test, run all matching tests, etc.
>
> http://code.google.com/p/googletest/
>
> I found a CMake macro that parses the source code and automatically 
> adds tests making it trivial to add new tests:
>
> macro(ADD_GOOGLE_TESTS executable)
>   foreach ( source ${ARGN} )
>     file(READ "${source}" contents)
>     string(REGEX MATCHALL "TEST_?F?\\(([A-Za-z_0-9 ,]+)\\)" 
> found_tests
> ${contents})
>     foreach(hit ${found_tests})
>       string(REGEX REPLACE ".*\\(([A-Za-z_0-9]+)[,
]*([A-Za-z_0-9]+)\\).*"
> "\\1.\\2" test_name ${hit})
>       add_test(${test_name} ${executable} --gtest_filter=${test_name}
> ${MI3CTestingDir})
>     endforeach(hit)
>   endforeach()
> endmacro()
>
>
> # Add all tests found in the source code, calling the executable to 
> run them add_google_tests ( ${EXECUTABLE_OUTPUT_PATH}/NoOp 
> ${mi3cTestSource})
>
> The main looks like this:
>
> #include <gtest/gtest.h>
>
> int main(int argc, char* argv[])
> {
>   testing::InitGoogleTest ( &argc, argv );
>   return RUN_ALL_TESTS();
> }
>
> and a test looks like this (we use MD5 hashes to test for pass/fail):
>
> #include <iostream>
>
> #include <mi3cImage.h>
> #include <mi3cImageLoader.h>
> #include <gtest/gtest.h>
> #include <NoOp.h>
>
> TEST(IO, LoadCT) {
>   mi3c::ImageLoader loader;
>   mi3c::Image::Pointer image = loader.setFilename ( dataFinder.getFile

> ( "CT.hdr" ) ).execute();
>   ASSERT_EQ ( "c1d43aaa5b991431a9daa1dc4b55dbb1", image->getMD5() ); }
>
>
> Fixtures are also supported to remove redundant code.
>
> Cheers,
> -dan
>
> On Mon, Dec 29, 2008 at 10:35 AM, Bill Lorensen 
> <bill.lorensen at gmail.com>
> wrote:
>>
>> I agree that we should look at a unit test facility.
>>
>> We should start a list of requirements. For example,
>>
>> 1) Must support all itk platforms.
>> 2) We must be able to distribute it with itk.
>> 3) It must fit within our test harness facility. Recall that we try 
>> to minimize the number of executables by combining large numbers of 
>> tests into FooTests.cxx files.
>> 4) It must be compatible with cmake/ctest/cdash. For example, a test 
>> must be able to "return EXIT_SUCCESS" and "return EXIT_FAILURE".
>> 5) It should not add complexity to an already complex testing
process.
>> 6) It must be compatible with itk's strict attention to compile 
>> warnings and dynamic memory anlysis. In other words, it must not 
>> produce warnings or purify defects.
>> 7) It should have a minimal source footprint.
>>
>> These are just a few requirements off the top of my head. I'll add 
>> them to the new page when I get a chance,
>>
>> Bill
>>
>> On Mon, Dec 29, 2008 at 10:47 AM, Luis Ibanez 
>> <luis.ibanez at kitware.com>
>> wrote:
>> >
>> > Hi Mathieu,
>> >
>> > Minor correction:
>> >
>> > The text of the UnitTestCpp license:
>> >
>> > https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/UnitT
>> > est++/COPYING
>> >
>> > doesn't correspond to a BSD license:
>> > http://www.opensource.org/licenses/bsd-license.php
>> >
>> > but to an MIT license:
>> > http://www.opensource.org/licenses/mit-license.php
>> >
>> > which is a technicality after all, since both licenses could be
combined
>> > without any conflict.   :-)
>> >
>> >
>> > -----
>> >
>> >
>> > Could you tell us more about your experience with UnitTestCpp  ?
>> >
>> > or if you prefer, could you add your comments to the Wiki page:
>> >
>> > http://www.itk.org/Wiki/Proposals:Increasing_ITK_Code_Coverage#Unit
>> > TestCpp
>> >
>> >
>> > The size of the package looks fine. It could fit neatly in 
>> > Insight/Utilities...
>> >
>> >
>> >  Thanks
>> >
>> >
>> >     Luis
>> >
>> >
>> > --------------------------
>> > Mathieu Malaterre wrote:
>> >>
>> >> On Mon, Dec 29, 2008 at 5:29 AM, Steve M. Robbins 
>> >> <steve at sumost.ca>
>> >> wrote:
>> >>
>> >>> Hello,
>> >>>
>> >>> On Wed, Dec 10, 2008 at 03:51:56PM -0500, Bill Lorensen wrote:
>> >>>
>> >>>
>> >>>> Given the success of "Adopt a Bug", maybe we should initiate a 
>> >>>> "Cover a Class".
>> >>>
>> >>> Now that I have some holiday time, I'd like to raise the issue of

>> >>> unit testing frameworks.
>> >>>
>> >>> The few test codes that I've looked at are all written out 
>> >>> long-hand, without benefit of a modern unit testing framework 
>> >>> such as CppUnit or Boost.Test.
>> >>
>> >>
>> >> ... or UnitTestCpp :)
>> >>
>> >> http://unittest-cpp.sourceforge.net/
>> >>
>> >> Which comes with a BSD like license ;)
>> >>
>> >>
>> >>
>> >> https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/Unit
>> >> Test++/COPYING
>> >>
>> >> I wrote a cmakelists.txt file for it:
>> >>
>> >> http://gdcm.svn.sf.net/viewvc/gdcm/Sandbox/UnitTest%2B%2B/
>> >>
>> >> src files are very lightweight "du -sk src" return 712
>> >>
>> >>
>> >> 2cts
>> >
>> _______________________________________________
>> Insight-developers mailing list
>> Insight-developers at itk.org
>> http://www.itk.org/mailman/listinfo/insight-developers
>
>
_______________________________________________
Insight-developers mailing list
Insight-developers at itk.org
http://www.itk.org/mailman/listinfo/insight-developers