[Insight-developers] RE: TestDriver Test

[Miller, James V (CRD)]

Bill,

Took me a while to figure out your macros and how arguments are passed to the 
functions. You threw me off having a variable ArgsMap which maps a string 
to a function pointer but does not use the map for function arguments.

So I would call this FunctionMap or TestMap.

So to summarize:

1) Each directory could have a single executable built that encapsulated all the 
   tests for the directory.
2) We would call the executable N times for N tests passing in the "name" of the 
   test to execute and the various args.

How does this affect how people put in tests now:

A) The test writer puts their test is a separate cxx file.
B) Instead of a main() function, they have a single function which is the 
   name of test.
C) The ADD_TEST line in CMakeLists.txt has the name of the test driver before
   the name of the specific test, i.e. instead of 
	ADD_TEST(itkAsinImageFilterAndAdaptorTest itkAsinImageFilterAndAdaptorTest)
   they would put
	ADD_TEST(itkAsinImageFilterAndAdaptorTest itkBasicFiltersTest 
	    itkAsinImageFilterAndAdaptorTest)
D) They add their cxx file to the test driver SOURCE_FILES list.
E) They edit the TestDriver.cxx file and add a call to RegisterTest()

Did I miss anything?

Also, people should realize they can still use the old style of adding tests as 
separate executables.  People could use that methodology when they first introduce
a test.  Once the test compiles on all platforms, we can move the test into
the TestDriver.

Finally, I wonder if there is a mechansism so the TestDriver.cxx does not need to 
be edited.  Could the RegisterTest() line be in the individual test cxx files? Then
we could skip step (E).






-----Original Message-----
From: William A. Hoffman [mailto:bill.hoffman@kitware.com]
Sent: Friday, March 01, 2002 5:15 PM
To: Lorensen, William E (CRD); Miller, James V (CRD); insight-Developers
Subject: TestDriver Test


Hi Folks-

Just to see if the testing process we discussed on the TCON was
worth doing at all, I went ahead and converted
just the tests in BasicFilters to my approach.  Here are the results
for this one directory:

The project load time in .NET went from 15 sec. to 8 sec. (1.2 ghz processor).
The number of projects went from 240 to 160.
The binary directory went from 240 megs to 70 megs.
The TestDriver executable was only 2.8 megs.
Changing a test and re-linking/compiling showed no noticeable difference in 
time.

The test driver program looked like this:

#include <map>
#include <string>

typedef int (*MainFunc)(int , char**);

std::map<std::string, MainFunc> ArgsMap;
#define REGISTER_TEST(test) \
extern int test(); \
ArgsMap[#test] = test;

int main(int ac, char** av)
{
   REGISTER_TEST(itkAsinImageFilterAndAdaptorTest );
   REGISTER_TEST(itkAcosImageFilterAndAdaptorTest );
   REGISTER_TEST(itkAtanImageFilterAndAdaptorTest );
....

  std::map<std::string, MainFunc>::iterator i = NoArgsMap.find(av[1]);
   // run the test
   std::map<std::string, MainFuncArgs>::iterator j = ArgsMap.find(av[1]);
   if(j != ArgsMap.end())
     {
     MainFuncArgs f = j->second;
     return (*f)(ac-1, av+1);
     }
}


The CMakeLists.txt file looked like this:

ADD_TEST(itkAsinImageFilterAndAdaptorTest itkBasicFiltersTest 
itkAsinImageFilterAndAdaptorTest)
ADD_TEST(itkAcosImageFilterAndAdaptorTest itkBasicFiltersTest 
itkAcosImageFilterAndAdaptorTest)
....
# a test with data arguments
ADD_TEST(itkConfidenceConnectedImageFilterTest itkBasicFiltersTest 
itkConfidenceConnectedImageFilterTest ${ITK_DATA_ROOT}/Input/cthead1.png 
${ITK_DATA_ROOT}/Baseline/BasicFilters/ConfidenceConnectedImageFilterTest.png)

SOURCE_FILES(BasicFilters_SRCS
itkAsinImageFilterAndAdaptorTest
itkAcosImageFilterAndAdaptorTest
...)



ADD_EXECUTABLE(itkBasicFiltersTest itkBasicFiltersTest.cxx BasicFilters_SRCS)