MantisBT - CDash
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0010684CDash[CMake] CTestpublic2010-05-06 15:322011-12-22 16:07
ReporterClinton Stimpson 
Assigned ToJulien Jomier 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version2.0 
Summary0010684: ctest not reporting submission failures
DescriptionI'm trying to debug why some submissions aren't showing up on cdash, but a "Submission successful" message is always given, but I don't see it on cdash.

But in general, I see a lack of error handling demonstrated below:

$ ctest -D ExperimentalSubmit
   Site: myhost
   Build name: myplatform
Submit files (using http)
   Using HTTP submit method
   Drop site:http://nohost.nodomain/cdash/submit.php?project=noproject [^]
   Submission successful

$ curl http://nohost.nodomain/cdash/submit.php?project=noproject [^]
curl: (6) Couldn't resolve host 'nohost.nodomain'

Then there's also the problem of having a valid host, but invalid project, where cdash returns "Not a valid project" but ctest says the submission was successful.

Then I've also seen its possible to get "no handler found" errors from cdash.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2010-05-06 15:32Clinton StimpsonNew Issue
2010-05-07 15:58David ColeNote Added: 0020650
2010-05-07 16:45Clinton StimpsonNote Added: 0020651
2010-12-14 18:33David ColeProjectCMake => CDash
2010-12-14 18:59David ColeAssigned To => Julien Jomier
2010-12-14 18:59David ColeStatusnew => assigned
2010-12-14 19:00David ColeNote Added: 0024122
2011-12-22 13:57David ColeNote Added: 0028047
2011-12-22 14:01David ColeNote Added: 0028048
2011-12-22 14:05David ColeNote Edited: 0028048bug_revision_view_page.php?bugnote_id=28048#r473
2011-12-22 14:07David ColeNote Edited: 0028048bug_revision_view_page.php?bugnote_id=28048#r474
2011-12-22 16:07Julien JomierNote Added: 0028050
2011-12-22 16:07Julien JomierStatusassigned => resolved
2011-12-22 16:07Julien JomierFixed in Version => 2.0
2011-12-22 16:07Julien JomierResolutionopen => fixed

Notes
(0020650)
David Cole   
2010-05-07 15:58   
Clinton,

What versions of ctest and cdash are you using with these results...?

There should definitely be an error message in ctest 2.8.1 and later for the nohost.nodomain url.

I have not tried submitting with an invalid project name, but a "non error" condition wouldn't surprise me terribly in that case...
(0020651)
Clinton Stimpson   
2010-05-07 16:45   
Ok, I see it some of it was fixed already.
But, having an invalid project still doesn't return an error.
(0024122)
David Cole   
2010-12-14 19:00   
The part of this bug that remains open is for CDash to return an error that ctest can recognize as a "failed submission" when somebody tries to submit to submit.php with an invalid project name.
(0028047)
David Cole   
2011-12-22 13:57   
The "no such project" condition is still a bug -- I just tried it with this project:

  https://github.com/dlrdave/SmallAndFast [^]

And pointed it to:

  "-DSAF_DROP_LOCATION:STRING=/submit.php?project=SmallAndFastZZZZZ"


And this is the end result of running "ctest -D Experimental" with ctest v2.8.6:


Submit files (using http)
   Using HTTP submit method
   Drop site:http://my.cdash.org/submit.php?project=SmallAndFastZZZZZ [^]
   Uploaded: /Users/davidcole/Dashboards/My Tests/SmallAndFast/b1/Testing/20111222-1854/Build.xml
   Uploaded: /Users/davidcole/Dashboards/My Tests/SmallAndFast/b1/Testing/20111222-1854/Configure.xml
   Uploaded: /Users/davidcole/Dashboards/My Tests/SmallAndFast/b1/Testing/20111222-1854/Test.xml
   Submission successful

No error reported. Because, of course, CDash can accept and process the xml file just fine, but then, there's no where to put the results. Moreover, it may be impossible to return an error in the async case, because the processing of the xml files happens later.

Not sure what the right fix is here.
(0028048)
David Cole   
2011-12-22 14:01   
(edited on: 2011-12-22 14:07)
Wait..... but the submit.php results for visiting http://my.cdash.org/submit.php?project=SmallAndFastZZZZZ [^] are a page with the text on it: "Not a valid project" -- if we made that match the error string that ctest is already looking for, then it would be an error that ctest reports.

Just changing the string to:
"ERROR: Not a valid project"

should cause ctest to recognize this as a failed submission.

Or using the more formal response xml should also work:

  "<cdash version=\"x.x.x\">\n"
  " <status>ERROR</status>\n"
  " <message>Not a valid project.</message>\n"
  "</cdash>\n"
(0028050)
Julien Jomier   
2011-12-22 16:07   
Thanks Dave! The error is now using the XML format. This fix also includes the error when CDash cannot connect to the database.