[CMake] Adding Tests - Suggestions needed

[Brad King]

Richard Wackerbarth wrote:
> The program was written to read part of its input from stdin. But I  
> don't see how to source stdin from a file when I use ADD_TEST.
> At the moment, I am using a template shell command.
> 
> CONFIGURE_FILE( ${CMAKE_CURRENT_SOURCE_DIR}/Test1Shell.in
>   ${CMAKE_CURRENT_BINARY_DIR}/Test1Shell.sh )
> ADD_TEST ( Test1
>   /bin/sh
>   ${CMAKE_CURRENT_BINARY_DIR}/Test1Shell.sh )
> 
> where Test1Shell.in is
> 
> #!/bin/sh
> 
> ${PROJECT_BINARY_DIR}/MyProgramBeingTested \
>   --using ${CMAKE_CURRENT_SOURCE_DIR}/Test1Database.txt \
>   -o ${CMAKE_CURRENT_BINARY_DIR}/Test1Output.txt \
>   <  ${CMAKE_CURRENT_SOURCE_DIR}/Test1TestData.txt
> 
> Is there a better way?

Currently there is no official CMake way to do this, but you could add a 
feature request here:

http://www.cmake.org/Bug

to support the redirection syntax.

If you want to do it cross-platform then you can look at the KWSys 
library that comes in CMake/Source/kwsys.  It is distributed under a 
very open license (just like CMake) and provides powerful cross-platform 
process execution tools.  We use it in several projects to drive tests. 
  Basically you would have one executable that is run for most tests 
that internally runs the real test.  This approach will let you create 
tests that have client/server connections through sockets or pipes 
pretty easily.

> Second question...
> In some tests, I want to verify that the output generated "matches" a  
> reference copy.
> If they were identical, "diff" would work. However, in this case,  there 
> are a few places where the date/time  or host name appear.
> Can someone suggest a comparison program that would be easy to use  and, 
> importantly, express the comparison in a simple manner.
> 
> At the moment, I am using sed/awk to edit the output and thereby  
> eliminate the variable part of the output before calling diff to do  the 
> comparison.

The above-mentioned approach of having a test-driver executable will let 
you implement this check yourself.  You can look at ParaView 
(www.paraview.org) for an example project that uses this approach.

-Brad