[CMake] CTest tests time out with CDash submission

[Christopher Mullins]

As a followup, I think I may have found part of the problem.
When I run $ ctest -N -VV -D ExperimentalMemCheck, I see test #29 listed as
follows:

29: MemCheck command: /usr/bin/valgrind "-q" "--tool=memcheck"
"--leak-check=yes" "--show-reachable=yes" "--workaround-gcc296-bugs=yes"
"--num-callers=50" "/home/crmullin/cppcheck/cppcheck/cppcheck" "-q" "--xml"
"--suppressions"
"/home/crmullin/TubeTK/Utilities/Cppcheck/TubeTKCppcheck.supp"
"--enable=style" "/home/crmullin/TubeTK/Applications"
"/home/crmullin/TubeTK/Base" "/home/crmullin/TubeTK/Utilities/Testing"
  Memory Check  #29: CppcheckStyle
Memory check command: /usr/bin/valgrind "-q" "--tool=memcheck"
"--leak-check=yes" "--show-reachable=yes" "--workaround-gcc296-bugs=yes"
"--num-callers=50"

Then if I run $ ctest -N -VV -R "CppcheckStyle", I see this same test
listed as follows:

30: Test command: /home/crmullin/cppcheck/cppcheck/cppcheck "-q" "--xml"
"--suppressions"
"/home/crmullin/TubeTK/Utilities/Cppcheck/TubeTKCppcheck.supp"
"--enable=style" "/home/crmullin/TubeTK/Applications"
"/home/crmullin/TubeTK/Base" "/home/crmullin/TubeTK/Utilities/Testing"
  Test #30: CppcheckStyle

The first method of invoking the test is running, but it runs incredibly
slowly, because it is primarily running valgrind, and not cppcheck!  The
second method is actually running cppcheck correctly, and thus it is
running more quickly.  Does anyone know how to instruct CTest to use the
correct executable?  I am having the same issue with the Cppcheck
integration in CalaTk, it appears to be maybe the same issue (although it's
hard to tell because Cppcheck is turned off by default in CalaTK, as we can
see on the CalaTK dashboards<http://open.cdash.org/index.php?project=calaTK>
).

Thanks for any help you can provide!


On Wed, Jan 2, 2013 at 3:57 PM, Christopher Mullins <
christopher.mullins at kitware.com> wrote:

> Hello CMake community,
>
> I am attempting to incorporate Cppcheck into the code analysis section of
> the TubeTK dashboard.  It's static code analysis, distinct from dynamic
> code analysis tools like Valgrind, but I'm simply incorporating the checks
> for Style, Performance, Portability, Information, and UnusedFunctions as
> additional tests in the same category.  Basically I'm trying to mimic what
> has been accomplished in the CalaTK project.<http://www.calatk.org/gitweb?p=calatk.git;a=blob;f=Code/Testing/CMakeLists.txt;h=2e900af70d5c6353b7dae38b5ebdb3282cbc4750;hb=HEAD#l1031>
>
> The issue I'm having is that the tests now run as such:
> TreCime ~/Projects/tubetk-cppcheck/TubeTK-Build $ time ctest -R "Cppcheck"
> Test project /home/crmullin/Projects/tubetk-cppcheck/TubeTK-Build
>     Start 30: CppcheckStyle
> 1/5 Test #30: CppcheckStyle ....................***Failed  Error regular
> expression found in output. Regex=[error file=] 77.12 sec
>     Start 31: CppcheckPerformance
> 2/5 Test #31: CppcheckPerformance ..............***Failed  Error regular
> expression found in output. Regex=[error file=] 74.77 sec
>     Start 32: CppcheckPortability
> 3/5 Test #32: CppcheckPortability ..............***Failed  Error regular
> expression found in output. Regex=[error file=] 75.78 sec
>     Start 33: CppcheckInformation
> 4/5 Test #33: CppcheckInformation ..............***Failed  Error regular
> expression found in output. Regex=[error file=] 75.37 sec
>     Start 34: CppcheckUnusedFunction
> 5/5 Test #34: CppcheckUnusedFunction ...........***Failed  Error regular
> expression found in output. Regex=[error file=] 75.02 sec
>
> 0% tests passed, 5 tests failed out of 5
>
> Total Test time (real) = 378.13 sec
>
> The following tests FAILED:
>  30 - CppcheckStyle (Failed)
>  31 - CppcheckPerformance (Failed)
>  32 - CppcheckPortability (Failed)
>  33 - CppcheckInformation (Failed)
>  34 - CppcheckUnusedFunction (Failed)
> Errors while running CTest
>
> real 6m18.135s
> user 6m17.444s
> sys 0m0.332s
>
>
> This is great.  It takes only a few minutes to complete, and fails when it
> should.  The same tests are *timing out* when I run $ make
> ExperimentalMemcheck:
> Start  27: ImageCompareCommand-Test6_fetchData
>  27/618 MemCheck  #27: ImageCompareCommand-Test6_fetchData
> ...............................................................................
>   Passed    3.50 sec
>         Start  28: ImageCompareCommand-Test6
>  28/618 MemCheck  #28: ImageCompareCommand-Test6
> .........................................................................................
>   Passed    3.08 sec
>         Start  29: CppcheckStyle
>  29/618 MemCheck  #29: CppcheckStyle
> .....................................................................................................***Timeout
> 1500.04 sec
>         Start  30: CppcheckPerformance
>  30/618 MemCheck  #30: CppcheckPerformance
> ...............................................................................................***Timeout
> 1500.04 sec
>         Start  31: CppcheckPortability
>  31/618 MemCheck  #31: CppcheckPortability
> ...............................................................................................***Timeout
> 1500.03 sec
>         Start  32: CppcheckInformation
>  32/618 MemCheck  #32: CppcheckInformation
> ...............................................................................................***Timeout
> 1500.04 sec
>
> During this time the process appears to be CPU-bound, so I don't think
> there is some blocking issue that I'm not seeing keeping the test idle.
>  Has anyone else encountered this problem?  Is this a CTest issue at all?
>
> Thanks for all your help,
> --
> Christopher Mullins
> R&D Engineer
> Kitware Inc.,
> 919.869.8871
>



-- 
Christopher Mullins
R&D Engineer
Kitware Inc.,
919.869.8871
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20130103/f299eac5/attachment.htm>