View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008674CMakeCTestpublic2009-03-05 09:032016-06-10 14:30
ReporterRoscoe A. Bartlett 
Assigned ToBill Hoffman 
PrioritynormalSeveritymajorReproducibilityhave not tried
StatusclosedResolutionmoved 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0008674: CTest Dev: Timed-out tests being reported as 'Failed' and 'Completed'
Description
I just noticed that CTest (and therefore CDash) is listing timed-out
tests as 'Failed' and 'Completed'. For example, the timed-out test
NOX_NOX_Sacado_FEApp_Brusselator_Example_MPI_4 shown at:

  http://trilinos-dev.sandia.gov/cdash/viewTest.php?buildid=2786 [^]

is listed as 'Failed' and 'Completed'. The only clue that I have that
the test timed out and did not fail for some other reason is that the
runtime is 600.04 seconds and I know that I set a global 600 second
time limit when I set up the test. I set up the nightly tests so I
know that 600 seconds is the time limit but no-one else would know
that and this this would confuse them to no end. In fact, I think I
saw this test failure a few days ago and I did not notice the 600
second runtime and I had no clue why the test was failing.

Note that this failed test is also shown as 'Completion Status' =
'Completed' at with 'Exit Value' = 0 at:

  http://trilinos-dev.sandia.gov/cdash/testDetails.php?test=25576&build=2786 [^]

I believe that this is the XML that was submitted for this failing
test (with some output skipped):








        <Test Status="failed">
                <Name>NOX_NOX_Sacado_FEApp_Brusselator_Example_MPI_4</Name>
                <Path>./packages/nox/examples/epetra/NOX_Sacado_FEApp</Path>
                <FullName>./packages/nox/examples/epetra/NOX_Sacado_FEApp/NOX_NOX_Sacado_FEApp_Brusselator_Example_MPI_4</FullName>
                <FullCommandLine>/usr/lib64/openmpi/1.2.7-gcc/bin/mpiexec -np 4 ./NOX_NOX_Sacado_FEApp_Brusselator_Example.exe -v</FullCommandLine>
                <Results>
                        <NamedMeasurement type="text/string" name="Exit Code"><Value>Timeout</Value></NamedMeasurement>
                        <NamedMeasurement type="text/string" name="Exit Value"><Value>0</Value></NamedMeasurement>
                        <NamedMeasurement type="numeric/double" name="Execution Time"><Value>600.037</Value></NamedMeasurement>
                        <NamedMeasurement type="text/string" name="Fail Reason"><Value>Required regular expression not found.Regex=[Test Passed
]</Value></NamedMeasurement>
                        <NamedMeasurement type="text/string" name="Completion Status"><Value>Completed</Value></NamedMeasurement>
                        <NamedMeasurement type="text/string" name="Command Line"><Value>/usr/lib64/openmpi/1.2.7-gcc/bin/mpiexec -np 4 ./NOX_NOX_Sacado_FEApp_Brusselator_Example.exe -v</Value></NamedMeasurement>
                        <Measurement>
                                <Value>DAT: library load failure: /usr/lib64/libdaplcma.so.1: undefined symbol: dat_registry_add_provider
--------------------------------------------------------------------------

WARNING: Failed to open "OpenIB-cma" [DAT_PROVIDER_NOT_FOUND:DAT_NAME_NOT_REGISTERED].
This may be a real error or it may be an invalid entry in the uDAPL
Registry which is contained in the dat.conf file. Contact your local
System Administrator to confirm the availability of the interfaces in
the dat.conf file.
--------------------------------------------------------------------------
DAT: library load failure: /usr/lib64/libdaplcma.so.1: undefined symbol: dat_registry_add_provider
--------------------------------------------------------------------------

...


************************************************************************

-- Parameters Passed to Nonlinear Solver --

     Nonlinear Solver = Line Search Based
     Direction ->
      Method = Newton
      Newton ->
       Forcing Term Method = Constant
       Rescue Bad Newton Solve = 1 [default]
       Linear Solver ->
        Aztec Solver = GMRES
        Compute Scaling Manually = 1 [default]
        Convergence Test = r0 [default]
        Max Age Of Prec = 1 [default]
        Max Iterations = 800 [unused]
        Orthogonalization = Classical [default]
        Output Frequency = 50
        Output Solver Details = 1 [default]
        Preconditioner = Ifpack
        Preconditioner Operator = Use Jacobian [default]
        Preconditioner Reuse Policy = Rebuild [default]
        RCM Reordering = Disabled [default]
        Size of Krylov Subspace = 300 [default]
        Throw Error on Prec Failure = 1 [default]
        Tolerance = 0.0001
        Zero Initial Guess = 0 [default]
     Line Search ->
      Method = Full Step
      Full Step ->
       Full Step = 1 [default]
     Printing ->
      MyPID = 0
      Output Information = 127
      Output Precision = 3
      Output Processor = 0
     Solver Options ->
      Status Test Check Type = Minimal [default]
</Value>
                        </Measurement>
                </Results>
                <Labels>
                        <Label>NOX</Label>
                </Labels>
        </Test>






I have marked this issue as 'major' because CTest seems to give no
indication whatsoever that this test is timing out.
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0039899)
Roscoe A. Bartlett (reporter)
2015-11-21 11:16

 Looks like this is fixed in newer versions of CDash, for example the one being worked on as part of the CASL ORNL Kitware project with an example here:

    https://casl-dev.ornl.gov/testing/testDetails.php?test=2812847&build=149236 [^]
(0041511)
Kitware Robot (administrator)
2016-06-10 14:27

 Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

 Issue History
Date Modified Username Field Change
2009-03-05 09:03 Roscoe A. Bartlett New Issue
2009-08-19 16:15 Bill Hoffman Status new => assigned
2009-08-19 16:15 Bill Hoffman Assigned To => Bill Hoffman
2015-11-21 11:16 Roscoe A. Bartlett Note Added: 0039899
2016-06-10 14:27 Kitware Robot Note Added: 0041511
2016-06-10 14:27 Kitware Robot Status assigned => resolved
2016-06-10 14:27 Kitware Robot Resolution open => moved
2016-06-10 14:30 Kitware Robot Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team