[Insight-developers] Test framework

[Bill Lorensen]

Dan,

Sounds cool. Can you add some info to:

http://www.itk.org/Wiki/Proposals:Increasing_ITK_Code_Coverage

Bill

On Mon, Dec 29, 2008 at 1:14 PM, Daniel Blezek <daniel.blezek at gmail.com> wrote:
> Hi Bill,
>
>   I would add the Google Test framework to the mix.  We've been using it
> extensively in our lab.  It meets 2, 3, 4, 5, 6, 7.  Not sure about platform
> support across _all_ ITK's platform, but I've never had a compile warning
> from it nor any memory leaks.  Given that they support several embedded
> platforms, I'd bet it compiles cleanly on all our platforms.  It's a very
> minimalist library, mostly implemented through macros and supports many of
> the nice ctest features, i.e. run every N'th test, run all matching tests,
> etc.
>
> http://code.google.com/p/googletest/
>
> I found a CMake macro that parses the source code and automatically adds
> tests making it trivial to add new tests:
>
> macro(ADD_GOOGLE_TESTS executable)
>   foreach ( source ${ARGN} )
>     file(READ "${source}" contents)
>     string(REGEX MATCHALL "TEST_?F?\\(([A-Za-z_0-9 ,]+)\\)" found_tests
> ${contents})
>     foreach(hit ${found_tests})
>       string(REGEX REPLACE ".*\\(([A-Za-z_0-9]+)[, ]*([A-Za-z_0-9]+)\\).*"
> "\\1.\\2" test_name ${hit})
>       add_test(${test_name} ${executable} --gtest_filter=${test_name}
> ${MI3CTestingDir})
>     endforeach(hit)
>   endforeach()
> endmacro()
>
>
> # Add all tests found in the source code, calling the executable to run them
> add_google_tests ( ${EXECUTABLE_OUTPUT_PATH}/NoOp ${mi3cTestSource})
>
> The main looks like this:
>
> #include <gtest/gtest.h>
>
> int main(int argc, char* argv[])
> {
>   testing::InitGoogleTest ( &argc, argv );
>   return RUN_ALL_TESTS();
> }
>
> and a test looks like this (we use MD5 hashes to test for pass/fail):
>
> #include <iostream>
>
> #include <mi3cImage.h>
> #include <mi3cImageLoader.h>
> #include <gtest/gtest.h>
> #include <NoOp.h>
>
> TEST(IO, LoadCT) {
>   mi3c::ImageLoader loader;
>   mi3c::Image::Pointer image = loader.setFilename ( dataFinder.getFile (
> "CT.hdr" ) ).execute();
>   ASSERT_EQ ( "c1d43aaa5b991431a9daa1dc4b55dbb1", image->getMD5() );
> }
>
>
> Fixtures are also supported to remove redundant code.
>
> Cheers,
> -dan
>
> On Mon, Dec 29, 2008 at 10:35 AM, Bill Lorensen <bill.lorensen at gmail.com>
> wrote:
>>
>> I agree that we should look at a unit test facility.
>>
>> We should start a list of requirements. For example,
>>
>> 1) Must support all itk platforms.
>> 2) We must be able to distribute it with itk.
>> 3) It must fit within our test harness facility. Recall that we try to
>> minimize the number of executables by combining large numbers of tests
>> into FooTests.cxx files.
>> 4) It must be compatible with cmake/ctest/cdash. For example, a test
>> must be able to "return EXIT_SUCCESS" and "return EXIT_FAILURE".
>> 5) It should not add complexity to an already complex testing process.
>> 6) It must be compatible with itk's strict attention to compile
>> warnings and dynamic memory anlysis. In other words, it must not
>> produce warnings or purify defects.
>> 7) It should have a minimal source footprint.
>>
>> These are just a few requirements off the top of my head. I'll add
>> them to the new page when I get a chance,
>>
>> Bill
>>
>> On Mon, Dec 29, 2008 at 10:47 AM, Luis Ibanez <luis.ibanez at kitware.com>
>> wrote:
>> >
>> > Hi Mathieu,
>> >
>> > Minor correction:
>> >
>> > The text of the UnitTestCpp license:
>> >
>> > https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/UnitTest++/COPYING
>> >
>> > doesn't correspond to a BSD license:
>> > http://www.opensource.org/licenses/bsd-license.php
>> >
>> > but to an MIT license:
>> > http://www.opensource.org/licenses/mit-license.php
>> >
>> > which is a technicality after all, since both licenses could be combined
>> > without any conflict.   :-)
>> >
>> >
>> > -----
>> >
>> >
>> > Could you tell us more about your experience with UnitTestCpp  ?
>> >
>> > or if you prefer, could you add your comments to the Wiki page:
>> >
>> > http://www.itk.org/Wiki/Proposals:Increasing_ITK_Code_Coverage#UnitTestCpp
>> >
>> >
>> > The size of the package looks fine. It could fit neatly in
>> > Insight/Utilities...
>> >
>> >
>> >  Thanks
>> >
>> >
>> >     Luis
>> >
>> >
>> > --------------------------
>> > Mathieu Malaterre wrote:
>> >>
>> >> On Mon, Dec 29, 2008 at 5:29 AM, Steve M. Robbins <steve at sumost.ca>
>> >> wrote:
>> >>
>> >>> Hello,
>> >>>
>> >>> On Wed, Dec 10, 2008 at 03:51:56PM -0500, Bill Lorensen wrote:
>> >>>
>> >>>
>> >>>> Given the success of "Adopt a Bug", maybe we should initiate a "Cover
>> >>>> a
>> >>>> Class".
>> >>>
>> >>> Now that I have some holiday time, I'd like to raise the issue of unit
>> >>> testing frameworks.
>> >>>
>> >>> The few test codes that I've looked at are all written out long-hand,
>> >>> without benefit of a modern unit testing framework such as CppUnit or
>> >>> Boost.Test.
>> >>
>> >>
>> >> ... or UnitTestCpp :)
>> >>
>> >> http://unittest-cpp.sourceforge.net/
>> >>
>> >> Which comes with a BSD like license ;)
>> >>
>> >>
>> >>
>> >> https://unittest-cpp.svn.sourceforge.net/svnroot/unittest-cpp/UnitTest++/COPYING
>> >>
>> >> I wrote a cmakelists.txt file for it:
>> >>
>> >> http://gdcm.svn.sf.net/viewvc/gdcm/Sandbox/UnitTest%2B%2B/
>> >>
>> >> src files are very lightweight "du -sk src" return 712
>> >>
>> >>
>> >> 2cts
>> >
>> _______________________________________________
>> Insight-developers mailing list
>> Insight-developers at itk.org
>> http://www.itk.org/mailman/listinfo/insight-developers
>
>