[CMake] ctest // fixtures and --repeat-until-fail <n>

[Sergei Nikulov]

>
> On Tue, Apr 16, 2019 at 10:07 PM Sergei Nikulov <sergey.nikulov at gmail.com> wrote:
>>
>> Hello All,
>>
>> Has anybody knows how FIXTURES_SETUP/FIXTURES_CLEANUP should work with
>> --repeat-until-fail <n> option?
>>
>> My expectation, if I requesting a test run for example 10 times, the
>> sequence should be as follows:
>>
>> fixture_setup, then test, then fixture_cleanup repeats 10 times
>>
>> But in practice, I've got following sequence
>>
>> 1. fixture_setup runs 10 times
>> 2. test runs 10 times
>> 3. fixture_cleanup runs 10 times
>>
>> Is it expected behavior?
>> I'm using ctest version 3.14.2
>
>
> The documentation for the --repeat-until-fail option says "Require each test to run <n> times without failing in order to pass". A fixture setup or cleanup test is still a test, so they will also be run <n> times. The logic that implements repeating tests sets up a counter on each test and it runs that test <n> times before marking that test as complete. This is why you see fixture_setup run 10 times, then test runs 10 times and lastly fixture_cleanup runs 10 times.
>

I understand that fixture is still a test.
To achieve repeatable behavior -R "test_continues_*" is enough.

But if it calls FIXTURE why not apply the specific behavior as
expected from fixture (init/teardown)?

-- 
Best Regards,
Sergei Nikulov