[CMake] ctest question

[Juan Sanchez]

Hi Alan,

I also have floating point results I want to account for.  I'm thinking
about writing a diff script for numerical results which uses an absolute
and relative error tolerance.  This would account for the difference in
transcendentals and other math functions between processors and math
libraries.  I always disable 80 bit extended precision on linux since
the results are non-deterministic with respect to compiler settings.

Regards,

Juan


Alan W. Irwin wrote:
> On 2007-09-24 10:05-0500 Juan Sanchez wrote:
> 
>> Hello Alan,
>>
>>> From your example, what in this statement that causes the test to run
>> when I type "make test"?
>>
>> ADD_TEST(my_first_test diff -q goldenfile testfile)
>>
>> All it says is to run diff.  How do I tell it to generate the testfile
>> from another executable?  How do I tell this executable to run only when
>> I type "make test" and not a moment before?
> 
> Hi Juan:
> 
> In the above simple example "diff" is run only when you run ctest (or I
> guess "make test" although I don't use that). So you could do something like
> 
> ADD_TEST(my_first_test ${CMAKE_CURRENT_BINARY_DIR}/create_testfile; diff -q goldenfile testfile)
> 
> subject to escaping of ";" which I can never get right until I experiment.
> (This general command-line approach of separating commands with ";" only
> works on Unix, I believe.)
> 
> Then the create_testfile executable is run at ctest time to create testfile
> and then diff is run immediately afterwards (which appears to be what you
> want).
> 
> A better approach would be to put everything you want including the diff
> into a configurable script, e.g.,
> 
> ADD_TEST(my_first_test ${CMAKE_CURRENT_BINARY_DIR}/test1.sh)
> 
> Note, in this case, the script is configured using CONFIGURE_FILE at
> cmake time (basically by substituting CMake-defined variables when needed),
> but run only at ctest time.
> 
> Our tests don't use diff (because postscript PLplot results are slightly
> platform/compiler version dependent because of floating-point rounding
> issues), but we do use a configurable scripting approach to generate our
> test plots, see
> http://plplot.svn.sourceforge.net/viewvc/plplot/trunk/test/CMakeLists.txt?view=log
> 
> I don't recommend you wade through _all_ of that CMake logic and bash script
> logic since it is so specific to our PLplot needs (and also its pretty
> voluminous/hierarchical since it deals with hundreds of test plots), but I
> have given you the above starting reference in case you have trouble
> configuring test scripts for yourself using CONFIGURE_FILE.
> 
> Alan
> __________________________
> Alan W. Irwin
> 
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
> 
> Programming affiliations with the FreeEOS equation-of-state implementation
> for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
> package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
> Linux Links project (loll.sf.net); and the Linux Brochure Project
> (lbproject.sf.net).
> __________________________
> 
> Linux-powered Science
> __________________________
> 
> 


-- 
Juan Sanchez
Juan.Sanchez at amd.com
800-538-8450 Ext. 54395
512-602-4395