[CMake] CxxTest failure output running unit tests

[Bill Hoffman]

Philip Lowman wrote:
> On Sun, Dec 7, 2008 at 5:44 PM, Tron Thomas <tron.thomas at verizon.net 
> <mailto:tron.thomas at verizon.net>> wrote:
> 
>     It would be nice if the original "test" and RUN_TESTS targets could
>     be configured this way without having to create and additional
>     target that replaces them.
> 
> 
> Patch attached here:
> http://public.kitware.com/Bug/view.php?id=8255
> 
> Here's a start at this that doesn't break existing behavior.  If the 
> user specifies a "--output-on-failure" flag to CTest or defines the 
> environment variable CTEST_OUTPUT_ON_FAILURE then when a test fails, 
> CTest will output whatever the test program outputted to the screen.  
> Usually this should be the test assertion(s) that failed.  Here's an 
> example of what it might look like when using CxxTest.
> 
> user at ubuntu:~/src/test_cmake_cxxtest/build$ echo "export 
> CTEST_OUTPUT_ON_FAILURE=1" >> ~/.bashrc
> user at ubuntu:~/src/test_cmake_cxxtest/build$ source ~/.bashrc
> user at ubuntu:~/src/test_cmake_cxxtest/build$ make test
> Running tests...
> Start processing tests
> Test project /home/user/src/test_cmake_cxxtest/build
>   1/  3 Testing unittest_addition ............   Passed
>   2/  3 Testing unittest_addition_fail .......
> *** Test Failed:
> Running 1 test
> In MyTestSuite::testAddition:
> /home/user/src/test_cmake_cxxtest/addition_failure.h:8: Error: Expected 
> (1 + 1 == 3), found (2 != 3)
> Failed 1 of 1 test
> Success rate: 0%
> 
> ***Failed
>   3/  3 Testing unittest_subtraction .........   Passed
> 
> 67% tests passed, 1 tests failed out of 3
> 
> The following tests FAILED:
>       2 - unittest_addition_fail (Failed)
> Errors while running CTest
> make: *** [test] Error 8
> 

I still think it looks ugly and hard to see the tests that failed.....

Does the  Testing/Temporary/LastTest.log file help at all?   So, if you 
run tests and there are failures, you can look at that file to see the 
detail.  Maybe if ctest did something like this:
The following tests FAILED:
        2 - unittest_addition_fail (Failed)
(See Testing/Temporary/LastTest.log for details.)


-Bill