[CMake] tests running out of order on cmake

[j s]

On Tue, Jun 28, 2011 at 9:57 AM, Michael Wild <themiwi at gmail.com> wrote:
> I agree that the information is not very discoverable if you've only
> read the doc about add_test(), but it is there:
>
> http://cmake.org/cmake/help/cmake-2-8-docs.html#command:set_tests_properties
>
> http://cmake.org/cmake/help/cmake-2-8-docs.html#section_PropertiesonTests

So the bug in the documentation is that:

add_test does not mention set_test_properties

set_test_properties documents only a few properties, but makes no
mention of the "PROPERTIES ON TESTS" section.

Another bug in the software, is that there is no way to specify that
the tests should run in order.  I tried the ctest -I option and it
didn't work.

Juan


>
> Michael
>
> On 06/28/2011 04:52 PM, j s wrote:
>> On Tue, Jun 28, 2011 at 9:46 AM, j s <j.s4403 at gmail.com> wrote:
>>> On Tue, Jun 28, 2011 at 9:33 AM, Michael Wild <themiwi at gmail.com> wrote:
>>>> If the tests depend on each other, you *must* tell CTest so, by setting
>>>> the DEPENDS property using the set_tests_properties() function.
>>>>
>>>> Normally, running tests in arbitrary order is considered to be a
>>>> feature, helping you to not to introduce accidental dependencies.
>>>
>>> I respectfully disagree.  It is a bug, which had been marked as major
>>> severity, as noted by the link in my previous email.  I have 85
>>> regressions, and the first ones run the fastest, so I immediately know
>>> if there is a problem.  Now that I am aware of set_test_properties,
>>> I'll set them for the tests that need them.  When not running tests in
>>> parallel, which is always, they need to run in the exact order
>>> specified.  It is only a feature, if there is a default option like
>>> schedule_sequential, which allows the program to work as I had come to
>>> expect over the past 6 years.
>>
>> It is also a bug since the man page for cmake 2.8.4 does not mention
>> setting dependencies with set_test_properties.
>>
>>>
>>> Juan
>>>
>>>>
>>>> Michael
>>>>
>>>> On 06/28/2011 04:16 PM, j s wrote:
>>>>> Can someone please let me know how to get the Mac OS X version of
>>>>> cmake 2.8.4 to run the tests in the order in which they are added?
>>>>> For some bizarre reason they are running out of order:
>>>>>
>>>>> Sample output (without actual test name).
>>>>>  1/85 Test #64: XXXXXXXXXXXXXXXX .............***Failed    0.01 sec
>>>>>       Start 63: XXXXXXXXXXXXXXXX
>>>>>  2/85 Test #63: XXXXXXXXXXXXXXXX..............   Passed    0.00 sec
>>>>>       Start 62: XXXXXXXXXXXXXXXX
>>>>>  3/85 Test #62: XXXXXXXXXXXXXXXX..............   Passed    0.00 sec
>>>>>       Start 61: XXXXXXXXXXXXXXXX
>>>>>
>>>>> This is preventing from verifying my recent changes on the mac.  The
>>>>> Ubuntu linux version does not have this issue.
>>>>>
>>>>> Juan
>>>>>
>>>>>
>>>>> On Sat, Jun 25, 2011 at 2:51 PM, j s <j.s4403 at gmail.com> wrote:
>>>>>> Unfortunately the problem came back.  I there a command line option to
>>>>>> fix this out of order execution feature.
>>>>>>
>>>>>> Juan
>>>>>>
>>>>>> On Sat, Jun 25, 2011 at 1:30 PM, j s <j.s4403 at gmail.com> wrote:
>>>>>>> I was able to update the cmake dmg from the website, and now my tests
>>>>>>> are running in the proper order.
>>>>>>>
>>>>>>> On Sat, Jun 25, 2011 at 1:23 PM, j s <j.s4403 at gmail.com> wrote:
>>>>>>>> How do I tell the macports version of cmake 2.8.1  on the mac to run
>>>>>>>> the tests in exactly the order they were originally specified?  I'm in
>>>>>>>> the process of updating macports right now to see if it goes back to
>>>>>>>> the sequential default in later versions?  man ctest reveals
>>>>>>>> --schedule-random, but no sequential option.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Juan
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.cmake.org/mailman/listinfo/cmake
>>>>
>>>
>
>