[CMake] CxxTest failure output running unit tests

[Bill Hoffman]

Philip Lowman wrote:

>> I still think it looks ugly and hard to see the tests that failed.....
> 
> Not all testing suites are going to be as verbose as CxxTest.  It's also not going to be any 
> easier on the eyes in the verbose logfile than it is on stdout.  In fact I would argue it's far 
> more readable on stdout because with the verbose option there is a lot more logging.
> 
I would say that most failed tests I have seen produce at least 10 if 
not 100 lines of output.
> Furthermore, the possibility remains that CxxTest  may be able to be 
> customized to simplify the output when running within another testing 
> system.
> 
>> Does the  Testing/Temporary/LastTest.log file help at all?   So, if you 
>> run tests and there are failures, you can look at that file to see the 
>> detail.  Maybe if ctest did something like this:
>> The following tests FAILED:
>>        2 - unittest_addition_fail (Failed)
>> (See Testing/Temporary/LastTest.log for details.)
> 
> LastTest.log is a nice thing to have but if the user has 800 unit tests I 
> think he'd prefer not to look through the entire logfile.
> 
If you had 800 tests and 10 or so lines for each failed tests, I would 
think having them scroll by on the screen would be bad.  I would rather 
see that there are some failures, and if I want to find them I can look 
in the file for them.
> How about giving the user the command to run to get information for the 
> tests that failed?
> 
> (Run "ctest --output-on-failure -N 2,5,9-11" for details about the failed 
> tests.) 
> 
> Obviously -N would also need to be implemented for this to work right (-I 
> is kinda bloated and doesn't support multiple ranges).
> 
We already have a -N and it does something different.  Maybe -I can be 
extended.  I would hate to bloat even more with an alternate command 
line option that did the same thing....

However, all that said, I suppose an option to output on failure would 
not be so bad as long as it is not the default.

-Bill