[CMake] Defining dependent compile and run tests using CMake/CTest

[Craig Scott]

On Mon, Nov 27, 2017 at 10:06 AM, Neil Carlson <neil.n.carlson at gmail.com>
wrote:

>
> On Sun, Nov 26, 2017 at 12:42 PM, Craig Scott <craig.scott at crascit.com>
> wrote:
>
>> Please post your CMakeLists.txt so we can see how you've got the fixtures
>> and tests defined.
>>
>
> Here's what I've got (I put the run test first to verify that the
> reordering takes place):
>
> add_test(NAME intel-20171126-run COMMAND intel-20171126)
> set_tests_properties(intel-20171126-run PROPERTIES
> PASS_REGULAR_EXPRESSION "pass")
> set_tests_properties(intel-20171126-run PROPERTIES FIXTURES_REQUIRED foo)
>
> add_executable(intel-20171126 EXCLUDE_FROM_ALL intel-20171126.f90)
> add_test(NAME intel-20171126
>     COMMAND ${CMAKE_COMMAND} --build . --target intel-20171126 --config
> $<CONFIGURATION>
>     WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
> set_tests_properties(intel-20171126 PROPERTIES FIXTURES_SETUP foo)
>
> After posting my question, I manually created the "executable"
> ('touch intel-20171126`)
> and re-ran ctest.  Sure enough, that time I didn't get the long message
> spew about
> being able to find the executable, but instead a 1-line message, which is
> what I had
> expected to see originally
>
>       Start  2: intel-20171126
>  1/13 Test  #2: intel-20171126 ...................***Failed    0.15 sec
>       Start  1: intel-20171126-run
> Failed test dependencies: intel-20171126
>  2/13 Test  #1: intel-20171126-run ...............***Not Run   0.00 sec
>
>
Thanks, that makes the situation clear. I've traced through the CTest
source and I think the logic can be improved. I've created an issue for
this in the bug tracker here:

https://gitlab.kitware.com/cmake/cmake/issues/17514

I'll look into a fix, but for now that bug report includes a workaround you
can use in the meantime.

On a side note, triggering parts of the build when running tests may seem
like a convenient solution, but be aware it has some less obvious gotchas.
If you are doing this with more than one test, multiple tests might try and
build simultaneously. If the things being built share any common
dependencies, the two builds may interfere with each other and overwrite
each other's files, etc. In the case of parallel builds, multiple build
invocations may result in excessive load due to too many tasks running at
once. This can cause out of memory issues if compiling or linking
particularly big files. If you want to be sure these things can't happen,
you may want to force such tests to run serially to guarantee that they are
the only build job:

set_tests_properties(intel-20171126 PROPERTIES RUN_SERIAL YES)

If you are confident that you can guarantee that you know all the test
cases that must not run simultaneously, you can use the RESOURCE_LOCK test
property instead to only make those specific tests run not at the same time
as each other. This will allow other harmless tests to run concurrently
with a build test case, leading to more efficient test execution when
parallel testing is enabled.

-- 
Craig Scott
Melbourne, Australia
https://crascit.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20171127/41856bf8/attachment-0001.html>