[CMake] CTest tests time out with CDash submission
Christopher Mullins
christopher.mullins at kitware.com
Thu Jan 3 15:13:42 EST 2013
Thanks so much! This appears to work, I really appreciate it. I
mistakenly thought ExperimentalMemCheck would simply run all the tests in
the category, not making any difference in which checker is used. How this
works is a little more clear now. In the future perhaps we'll reopen the
discussion of having a separate category for static code analysis.
On Thu, Jan 3, 2013 at 3:03 PM, David Cole <david.cole at kitware.com> wrote:
> Why not just run "make Experimental" instead?
>
> If your tests already run the checkers that you want, then just running
> the tests is sufficient. You don't need to run make ExperimentalMemCheck if
> your checkers run as part of your test suite...
>
> Running the MemCheck runs all tests via the memory checker.
>
>
>
> On Thu, Jan 3, 2013 at 2:47 PM, Christopher Mullins <
> christopher.mullins at kitware.com> wrote:
>
>> As a followup, I think I may have found part of the problem.
>> When I run $ ctest -N -VV -D ExperimentalMemCheck, I see test #29 listed
>> as follows:
>>
>> 29: MemCheck command: /usr/bin/valgrind "-q" "--tool=memcheck"
>> "--leak-check=yes" "--show-reachable=yes" "--workaround-gcc296-bugs=yes"
>> "--num-callers=50" "/home/crmullin/cppcheck/cppcheck/cppcheck" "-q" "--xml"
>> "--suppressions"
>> "/home/crmullin/TubeTK/Utilities/Cppcheck/TubeTKCppcheck.supp"
>> "--enable=style" "/home/crmullin/TubeTK/Applications"
>> "/home/crmullin/TubeTK/Base" "/home/crmullin/TubeTK/Utilities/Testing"
>> Memory Check #29: CppcheckStyle
>> Memory check command: /usr/bin/valgrind "-q" "--tool=memcheck"
>> "--leak-check=yes" "--show-reachable=yes" "--workaround-gcc296-bugs=yes"
>> "--num-callers=50"
>>
>> Then if I run $ ctest -N -VV -R "CppcheckStyle", I see this same test
>> listed as follows:
>>
>> 30: Test command: /home/crmullin/cppcheck/cppcheck/cppcheck "-q" "--xml"
>> "--suppressions"
>> "/home/crmullin/TubeTK/Utilities/Cppcheck/TubeTKCppcheck.supp"
>> "--enable=style" "/home/crmullin/TubeTK/Applications"
>> "/home/crmullin/TubeTK/Base" "/home/crmullin/TubeTK/Utilities/Testing"
>> Test #30: CppcheckStyle
>>
>> The first method of invoking the test is running, but it runs incredibly
>> slowly, because it is primarily running valgrind, and not cppcheck! The
>> second method is actually running cppcheck correctly, and thus it is
>> running more quickly. Does anyone know how to instruct CTest to use the
>> correct executable? I am having the same issue with the Cppcheck
>> integration in CalaTk, it appears to be maybe the same issue (although it's
>> hard to tell because Cppcheck is turned off by default in CalaTK, as we can
>> see on the CalaTK dashboards<http://open.cdash.org/index.php?project=calaTK>
>> ).
>>
>> Thanks for any help you can provide!
>>
>>
>> On Wed, Jan 2, 2013 at 3:57 PM, Christopher Mullins <
>> christopher.mullins at kitware.com> wrote:
>>
>>> Hello CMake community,
>>>
>>> I am attempting to incorporate Cppcheck into the code analysis section
>>> of the TubeTK dashboard. It's static code analysis, distinct from dynamic
>>> code analysis tools like Valgrind, but I'm simply incorporating the checks
>>> for Style, Performance, Portability, Information, and UnusedFunctions as
>>> additional tests in the same category. Basically I'm trying to mimic what
>>> has been accomplished in the CalaTK project.<http://www.calatk.org/gitweb?p=calatk.git;a=blob;f=Code/Testing/CMakeLists.txt;h=2e900af70d5c6353b7dae38b5ebdb3282cbc4750;hb=HEAD#l1031>
>>>
>>> The issue I'm having is that the tests now run as such:
>>> TreCime ~/Projects/tubetk-cppcheck/TubeTK-Build $ time ctest -R
>>> "Cppcheck"
>>> Test project /home/crmullin/Projects/tubetk-cppcheck/TubeTK-Build
>>> Start 30: CppcheckStyle
>>> 1/5 Test #30: CppcheckStyle ....................***Failed Error regular
>>> expression found in output. Regex=[error file=] 77.12 sec
>>> Start 31: CppcheckPerformance
>>> 2/5 Test #31: CppcheckPerformance ..............***Failed Error regular
>>> expression found in output. Regex=[error file=] 74.77 sec
>>> Start 32: CppcheckPortability
>>> 3/5 Test #32: CppcheckPortability ..............***Failed Error regular
>>> expression found in output. Regex=[error file=] 75.78 sec
>>> Start 33: CppcheckInformation
>>> 4/5 Test #33: CppcheckInformation ..............***Failed Error regular
>>> expression found in output. Regex=[error file=] 75.37 sec
>>> Start 34: CppcheckUnusedFunction
>>> 5/5 Test #34: CppcheckUnusedFunction ...........***Failed Error regular
>>> expression found in output. Regex=[error file=] 75.02 sec
>>>
>>> 0% tests passed, 5 tests failed out of 5
>>>
>>> Total Test time (real) = 378.13 sec
>>>
>>> The following tests FAILED:
>>> 30 - CppcheckStyle (Failed)
>>> 31 - CppcheckPerformance (Failed)
>>> 32 - CppcheckPortability (Failed)
>>> 33 - CppcheckInformation (Failed)
>>> 34 - CppcheckUnusedFunction (Failed)
>>> Errors while running CTest
>>>
>>> real 6m18.135s
>>> user 6m17.444s
>>> sys 0m0.332s
>>>
>>>
>>> This is great. It takes only a few minutes to complete, and fails when
>>> it should. The same tests are *timing out* when I run $ make
>>> ExperimentalMemcheck:
>>> Start 27: ImageCompareCommand-Test6_fetchData
>>> 27/618 MemCheck #27: ImageCompareCommand-Test6_fetchData
>>> ...............................................................................
>>> Passed 3.50 sec
>>> Start 28: ImageCompareCommand-Test6
>>> 28/618 MemCheck #28: ImageCompareCommand-Test6
>>> .........................................................................................
>>> Passed 3.08 sec
>>> Start 29: CppcheckStyle
>>> 29/618 MemCheck #29: CppcheckStyle
>>> .....................................................................................................***Timeout
>>> 1500.04 sec
>>> Start 30: CppcheckPerformance
>>> 30/618 MemCheck #30: CppcheckPerformance
>>> ...............................................................................................***Timeout
>>> 1500.04 sec
>>> Start 31: CppcheckPortability
>>> 31/618 MemCheck #31: CppcheckPortability
>>> ...............................................................................................***Timeout
>>> 1500.03 sec
>>> Start 32: CppcheckInformation
>>> 32/618 MemCheck #32: CppcheckInformation
>>> ...............................................................................................***Timeout
>>> 1500.04 sec
>>>
>>> During this time the process appears to be CPU-bound, so I don't think
>>> there is some blocking issue that I'm not seeing keeping the test idle.
>>> Has anyone else encountered this problem? Is this a CTest issue at all?
>>>
>>> Thanks for all your help,
>>> --
>>> Christopher Mullins
>>> R&D Engineer
>>> Kitware Inc.,
>>> 919.869.8871
>>>
>>
>>
>>
>> --
>> Christopher Mullins
>> R&D Engineer
>> Kitware Inc.,
>> 919.869.8871
>>
>> --
>>
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.cmake.org/mailman/listinfo/cmake
>>
>
>
--
Christopher Mullins
R&D Engineer
Kitware Inc.,
919.869.8871
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20130103/49859b03/attachment.htm>
More information about the CMake
mailing list