<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
We've been using cdash 2.2.2 for quite a while but the server
running it is going to be decommissioned so I thought I'd take the
opportunity to upgrade to 2.4.0 (although it says 2.5.0 on the
bottom of the dashboard). So I clone the repository and switch to
the <i>v2.4.0-prebuilt</i> tag and point the webserver right to the
clone. I've got two problems I've been fighting since the install...<br>
<br>
Problem one, last night all our regression tests completed
successfully and all submitted to cdash but one of the expected
builds is absent from the database. The submission reported as
successfully completed by the test execution platform, I can see, in
the job's log, the successful submission of the XML files to my
cdash site, I can see the submitted files associated with the job in
the <i>CDash/backup</i> directory, but I see nothing for that job
in the database and therefore, of course, the dashboard is showing
an empty line for that particular test with <i>Expected build</i>
in the <b>Start Time</b> column. It appears the job is being
successfully submitted but not inserted into the database.<br>
<br>
Problem two, false negatives in email. The first time this happened
I had inadvertently enabled test timing and since it was starting
from scratch the times were varying. Just looking at the dashboard
there was no indication of this until I turned on the <i>Advanced
View. </i>Then I could see a red column for timings and I figured
out what was going on and disabled test timing. However, that didn't
completely eliminate the problem. Again, last night all our
regression tests passed and aside from the aforementioned missing
build in the database, all the columns are green. But I got the
following email:<br>
<br>
<pre class="moz-quote-pre" wrap="">A submission to CDash for the project Charon has failing tests.
You have been identified as one of the authors who have checked in changes that are part of this submission or you are listed in the default contact list.
Details on the submission can be found at <a class="moz-txt-link-freetext" href="http://verne.srn.sandia.gov/cdash/buildSummary.php?buildid=252">http://cdashmachine.my.url/cdash/buildSummary.php?buildid=252</a>
Project: Charon
Site: CEE Build Farm
Build Name: Linux-x86_64-RHEL_6.x-SEMS_OpenMPI_1.10_Intel_17.x-DBG
Build Time: 2018-10-19T01:00:16 UTC
Type: Nightly
Tests failing: 3
-CDash on cdashmachine.my.url
</pre>
I follow the link and I see:<br>
<br>
<br>
<table class="dart">
<tbody>
<tr class="table-heading">
<th colspan="3">Current Build</th>
</tr>
<tr class="table-heading">
<th>Stage</th>
<th>Errors</th>
<th>Warnings</th>
</tr>
<tr class="tr-odd">
<td> <a class="ng-scope"> <b>Update</b> </a> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewUpdate.php?buildid=252"> 0 </a>
</b> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewUpdate.php?buildid=252"> 0 </a>
</b> </td>
</tr>
<tr class="tr-even">
<td> <a> <b>Configure</b> </a> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewConfigure.php?buildid=252">
0 </a> </b> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewConfigure.php?buildid=252">
0 </a> </b> </td>
</tr>
<tr class="tr-odd">
<td> <a> <b>Build</b> </a> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewBuildError.php?buildid=252">
0 </a> </b> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewBuildError.php?type=&buildid=252">
0 </a> </b> </td>
</tr>
<tr class="tr-even">
<td><a><b>Test</b></a></td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewTest.php?onlyfailed&buildid=252">
0 </a> </b> </td>
<td class="normal" align="right"> <b> <a class="ng-binding"
href="http://verne.srn.sandia.gov/cdash/viewTest.php?onlynotrun&buildid=252">
0 </a> </b></td>
</tr>
</tbody>
</table>
<br>
So those 3 failing tests are showing up here. Now if I look at the
bottom of the page, and I select "View Tests Summary" I get a clue<br>
<br>
<b>107 passed, 0 failed, 3 timed out, 0 not run.<br>
<br>
</b>I don't know where that <i>"3 timed out"</i> is coming from and
they don't show up in the table associated with this test summary
below that. This test suite does have 110 tests, but 3 of them are
excluded from this particular platform/compiler combination via a
"-LE debugexclude" and tagging those three tests with a
"debugexclude". I can't for the life of me figure out why it thinks
three tests have "timed out".<br>
<br>
One thing that could be an issue, maybe related to both my problems,
is uniqueness. Does "BuildName" have to be unique? Typically, in our
case, only the combination of "BuildName" and "Name" are unique.
Also we use jenkins for some of our testing and the platform/site
that those tests run on isn't stable. Each night the tests may run
on a different machine in the jenkins build farm. In those tests we
set the value of "Name" by using SET(CTEST_SITE <name>). Maybe
I'm just grasping at straws here.<br>
<br>
Lastly, almost nothing has changed in how we're doing things. I
simply changed our submissions from the old 2.2.2 CDash, which was
working fine, to my new install of 2.4.0 on a new platform. <br>
<br>
Hopefully someone can offer some suggestions on my likely overly
detailed description, or at least how to start to debug it.<br>
<br>
Thanks,<br>
Gary<br>
<br>
<b></b>
</body>
</html>