MantisBT - CMake
View Issue Details
0009648CMakeCTestpublic2009-10-02 11:362010-12-14 18:49
Derek Bruening 
Zach Mullen 
normalminoralways
closedfixed 
CMake-2-6 
CMake-2-8 
0009648: ctest in non-verbose mode prints "The following tests FAILED:" with no subsequent list of tests
Running ctest using extended scripting, if any tests fail, the ctest run prints this message:

  The following tests FAILED:

But it doesn't follow that up with a list of tests. Better to print nothing unless -V is specified; or, to include the failing tests.

I tried cmake-2.8.0-rc2-Linux-i386 and it has the same behavior.
No tags attached.
tgz bug9648.tgz (847) 2009-10-05 15:59
https://public.kitware.com/Bug/file/2527/bug9648.tgz
Issue History
2009-10-02 11:36Derek BrueningNew Issue
2009-10-02 11:40Bill HoffmanStatusnew => assigned
2009-10-02 11:40Bill HoffmanAssigned To => Zach Mullen
2009-10-02 11:49Bill HoffmanNote Added: 0017900
2009-10-02 12:03Derek BrueningNote Added: 0017903
2009-10-05 11:00Zach MullenNote Added: 0017946
2009-10-05 13:00Zach MullenStatusassigned => resolved
2009-10-05 13:00Zach MullenFixed in Version => CMake-2-8
2009-10-05 13:00Zach MullenResolutionopen => fixed
2009-10-05 15:58Derek BrueningNote Added: 0017962
2009-10-05 15:58Derek BrueningStatusresolved => feedback
2009-10-05 15:58Derek BrueningResolutionfixed => reopened
2009-10-05 15:59Derek BrueningFile Added: bug9648.tgz
2009-10-05 16:03Derek BrueningNote Added: 0017963
2009-10-05 18:32Bill HoffmanNote Added: 0017965
2009-10-06 17:58Derek BrueningNote Added: 0018000
2009-10-20 12:11Zach MullenNote Added: 0018124
2009-10-29 09:33Zach MullenStatusfeedback => resolved
2009-10-29 09:33Zach MullenResolutionreopened => fixed
2010-12-14 18:49David ColeNote Added: 0024079
2010-12-14 18:49David ColeStatusresolved => closed

Notes
(0017900)
Bill Hoffman   
2009-10-02 11:49   
I can not reproduce this:
$ /cygdrive/c/Program\ Files/CMake\ 2.8/bin/ctest.exe
Test project C:/hoffman/My Builds/CMake/Tests/Tutorial/Step7/b
        Start 1: TutorialRuns
1/9 Test #1: TutorialRuns ..................... Passed 0.03 sec
        Start 2: TutorialUsage
2/9 Test 0000002: TutorialUsage .................... Passed 0.03 sec
        Start 3: TutorialComp4
3/9 Test 0000003: TutorialComp4 ....................***Failed Required regular expression not found.Regex=[4 is 2xx
] 0.03 sec
        Start 4: TutorialComp9
4/9 Test 0000004: TutorialComp9 .................... Passed 0.05 sec
        Start 5: TutorialComp5
5/9 Test 0000005: TutorialComp5 .................... Passed 0.03 sec
        Start 6: TutorialComp7
6/9 Test 0000006: TutorialComp7 .................... Passed 0.00 sec
        Start 7: TutorialComp25
7/9 Test 0000007: TutorialComp25 ................... Passed 0.02 sec
        Start 8: TutorialComp-25
8/9 Test 0000008: TutorialComp-25 .................. Passed 0.00 sec
        Start 9: TutorialComp0.0001
9/9 Test #9: TutorialComp0.0001 ............... Passed 0.02 sec

89% tests passed, 1 tests failed out of 9

Total Test time (real) = 0.22 sec

The following tests FAILED:
          3 - TutorialComp4 (Failed)
Errors while running CTest

I tested on both windows and linux.

Do you have a sample project tree that shows this problem?
(0017903)
Derek Bruening   
2009-10-02 12:03   
Like I said, this is when using ctest with extended scripting (I believe that's what you call it?), not when just running "ctest" in a build directory. I.e., running "ctest -S <ctest script that uses ctest_test(), etc.>". This file is an example:

  http://code.google.com/p/dynamorio/source/browse/trunk/suite/runsuite.cmake [^]

Here's some example output:

  > ctest -S /work/opensource/withwiki/trunk/suite/runsuite.cmake
  
  The following tests FAILED:
  No tests were found!!!
  No tests were found!!!
  No tests were found!!!
  No tests were found!!!
  No tests were found!!!
  No tests were found!!!

Ignore the "No tests were found", in this short run we have some build configurations that we just want to ensure still build and we don't run any tests for them -- not relevant to this issue.
(0017946)
Zach Mullen   
2009-10-05 11:00   
A lot of our tests are using the -S option, and none of them show this result. I went into the CMakeLists file used to build one such test and commented out all of the add_test lines so that no tests were found, but even then I did not the see the "following tests FAILED:" message. I attempted running with the script you linked but it obviously won't work outside of its project... Even so, I did not see the FAILED message. I'm a bit stumped as to how to recreate this.
(0017962)
Derek Bruening   
2009-10-05 15:58   
I stated that the same bug exists in 2.8 so I don't know why this
case was resolved. I have created a tiny sample project that
shows the error.
(0017963)
Derek Bruening   
2009-10-05 16:03   
I just attached a tiny project that shows the error.
cmake 2.8 also prints out "Error in read script" but gives no further information.

% mkdir tmp
% cd tmp
% tar xzf ../bug9648.tgz
% mkdir suite
% cd suite
% /extsw/pkgs/cmake/cmake-2.8.0-rc2-Linux-i386/bin/ctest -S ../src/runtests.cmake

The following tests FAILED:
Error in read script: /work/bugs-cmake/bug9648/tmp/src/runtests.cmake

% ctest -S ../src/runtests.cmake

The following tests FAILED:

% ctest --version
ctest version 2.6-patch 4
(0017965)
Bill Hoffman   
2009-10-05 18:32   
I think it was closed because Zach checked in a fix:
Index: cmCTestTestHandler.cxx
===================================================================
RCS file: /cvsroot/CMake/CMake/Source/CTest/cmCTestTestHandler.cxx,v
retrieving revision 1.122
retrieving revision 1.123
diff -C 2 -d -r1.122 -r1.123
*** cmCTestTestHandler.cxx 2 Oct 2009 19:30:01 -0000 1.122
--- cmCTestTestHandler.cxx 5 Oct 2009 16:47:09 -0000 1.123
***************
*** 597,601 ****
        {
        cmGeneratedFileStream ofs;
! cmCTestLog(this->CTest, ERROR_MESSAGE, std::endl
                   << "The following tests FAILED:" << std::endl);
        this->StartLogFile("TestsFailed", ofs);
--- 597,601 ----
        {
        cmGeneratedFileStream ofs;
! cmCTestLog(this->CTest, HANDLER_OUTPUT, std::endl
                   << "The following tests FAILED:" << std::endl);
        this->StartLogFile("TestsFailed", ofs);
(0018000)
Derek Bruening   
2009-10-06 17:58   
ah, ok, that wasn't clear.
any idea what the "Error in read script" is referring to?
(0018124)
Zach Mullen   
2009-10-20 12:11   
That message gets output when an error condition is reported by CTest, which happens if any test fails or cannot be run.
(0024079)
David Cole   
2010-12-14 18:49   
Closing bugs that have been resolved for more than 3 months without any further updates.