[CMake] ctest question

[Juan Sanchez]

Awesome.  I love that guys work on bibtex and bibtools.

Juan

Mathieu MARACHE wrote:
> 2007/9/25, Juan Sanchez <Juan.Sanchez at amd.com>:
>> Hi Alan,
>>
>> I also have floating point results I want to account for.  I'm thinking
>> about writing a diff script for numerical results which uses an absolute
>> and relative error tolerance.  This would account for the difference in
>> transcendentals and other math functions between processors and math
>> libraries.  I always disable 80 bit extended precision on linux since
>> the results are non-deterministic with respect to compiler settings.
> 
> diff with numerical precision concerns is ndiff :
> http://www.math.utah.edu/~beebe/software/ndiff
> 
> HTH
> 
>> Regards,
>>
>> Juan
>>
>>
>> Alan W. Irwin wrote:
>>> On 2007-09-24 10:05-0500 Juan Sanchez wrote:
>>>
>>>> Hello Alan,
>>>>
>>>>> From your example, what in this statement that causes the test to run
>>>> when I type "make test"?
>>>>
>>>> ADD_TEST(my_first_test diff -q goldenfile testfile)
>>>>
>>>> All it says is to run diff.  How do I tell it to generate the testfile
>>>> from another executable?  How do I tell this executable to run only when
>>>> I type "make test" and not a moment before?
>>> Hi Juan:
>>>
>>> In the above simple example "diff" is run only when you run ctest (or I
>>> guess "make test" although I don't use that). So you could do something like
>>>
>>> ADD_TEST(my_first_test ${CMAKE_CURRENT_BINARY_DIR}/create_testfile; diff -q goldenfile testfile)
>>>
>>> subject to escaping of ";" which I can never get right until I experiment.
>>> (This general command-line approach of separating commands with ";" only
>>> works on Unix, I believe.)
>>>
>>> Then the create_testfile executable is run at ctest time to create testfile
>>> and then diff is run immediately afterwards (which appears to be what you
>>> want).
>>>
>>> A better approach would be to put everything you want including the diff
>>> into a configurable script, e.g.,
>>>
>>> ADD_TEST(my_first_test ${CMAKE_CURRENT_BINARY_DIR}/test1.sh)
>>>
>>> Note, in this case, the script is configured using CONFIGURE_FILE at
>>> cmake time (basically by substituting CMake-defined variables when needed),
>>> but run only at ctest time.
>>>
>>> Our tests don't use diff (because postscript PLplot results are slightly
>>> platform/compiler version dependent because of floating-point rounding
>>> issues), but we do use a configurable scripting approach to generate our
>>> test plots, see
>>> http://plplot.svn.sourceforge.net/viewvc/plplot/trunk/test/CMakeLists.txt?view=log
>>>
>>> I don't recommend you wade through _all_ of that CMake logic and bash script
>>> logic since it is so specific to our PLplot needs (and also its pretty
>>> voluminous/hierarchical since it deals with hundreds of test plots), but I
>>> have given you the above starting reference in case you have trouble
>>> configuring test scripts for yourself using CONFIGURE_FILE.
>>>
>>> Alan
>>> __________________________
>>> Alan W. Irwin
>>>
>>> Astronomical research affiliation with Department of Physics and Astronomy,
>>> University of Victoria (astrowww.phys.uvic.ca).
>>>
>>> Programming affiliations with the FreeEOS equation-of-state implementation
>>> for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
>>> package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
>>> Linux Links project (loll.sf.net); and the Linux Brochure Project
>>> (lbproject.sf.net).
>>> __________________________
>>>
>>> Linux-powered Science
>>> __________________________
>>>