[CMake] Problem with CTest and configuration specific binary paths

Tue Aug 24 15:04:17 EDT 2010

Thanks David. I'll stick to the add_test( NAME ... COMMAND ... ) syntax then.

I tried the script to force CTEST_CONFIGURATION_TYPE to be set, but it
didn't seem to do anything here. When I tried to debug it, the regular
'message(..)' command didn't seem to print, even with ctest -VV. Also,
when i was changing the file it was giving some strange parse errors,
is the syntax different from other cmake files?

Thanks,

Iman

On Mon, Aug 23, 2010 at 9:42 AM, David Cole <david.cole at kitware.com> wrote:
> The behavior you're seeing is intentional.
> In CMake itself we do the following 3 things to force a configuration even
> if the user does not run ctest with -C :
>
> (1) In the main CMakeLists.txt file:
>
> # Set up test-time configuration.
>
> SET_DIRECTORY_PROPERTIES(PROPERTIES
>
>   TEST_INCLUDE_FILE "${CMake_BINARY_DIR}/Tests/EnforceConfig.cmake")
>
> (2) Then in the Tests/CMakeLists file, we configure EnforceConfig.cmake:
>
> CONFIGURE_FILE(${CMake_SOURCE_DIR}/Tests/EnforceConfig.cmake.in
>
>                ${CMake_BINARY_DIR}/Tests/EnforceConfig.cmake @ONLY)
>
> (3) Then, in EnforceConfig.cmake.in:
>
> # Choose a configuration with which to drive CTest tests.
>
> IF(CTEST_CONFIGURATION_TYPE)
>
>   SET(CTestTest_CONFIG "${CTEST_CONFIGURATION_TYPE}")
>
> ELSE(CTEST_CONFIGURATION_TYPE)
>
>   SET(CTestTest_CONFIG "@CTestTest_CONFIG@")
>
> ENDIF(CTEST_CONFIGURATION_TYPE)
>
> # Choose a configuration that was built if none is given.
>
> IF(NOT CTEST_CONFIGURATION_TYPE)
>
>   SET(CTEST_CMD "@CMAKE_CTEST_COMMAND@@CMAKE_EXECUTABLE_SUFFIX@")
>
>   GET_FILENAME_COMPONENT(CTEST_DIR "${CTEST_CMD}" PATH)
>
>   GET_FILENAME_COMPONENT(CTEST_EXE "${CTEST_CMD}" NAME)
>
>   FOREACH(cfg Release Debug MinSizeRel RelWithDebInfo)
>
>     IF(NOT CTEST_CONFIGURATION_TYPE)
>
>       IF(EXISTS "${CTEST_DIR}/${cfg}/${CTEST_EXE}")
>
>         SET(CTEST_CONFIGURATION_TYPE ${cfg})
>
>       ENDIF(EXISTS "${CTEST_DIR}/${cfg}/${CTEST_EXE}")
>
>     ENDIF(NOT CTEST_CONFIGURATION_TYPE)
>
>   ENDFOREACH(cfg)
>
>   IF(NOT CTEST_CONFIGURATION_TYPE)
>
>     SET(CTEST_CONFIGURATION_TYPE NoConfig)
>
>   ENDIF(NOT CTEST_CONFIGURATION_TYPE)
>
>   MESSAGE("Guessing configuration ${CTEST_CONFIGURATION_TYPE}")
>
> ENDIF(NOT CTEST_CONFIGURATION_TYPE)
>
> @TEST_HOME_ENV_CODE@
>
> The net result is that EnforceConfig.cmake gets included at ctest time
> because of the TEST_INCLUDE_FILE directory property. And... even if the user
> does not use -C, CTEST_CONFIGURATION_TYPE gets set to a reasonable value at
> ctest time.
>
> HTH,
> David
>
> On Mon, Aug 23, 2010 at 9:21 AM, Iman Brouwer <iman.brouwer at gmail.com>
> wrote:
>>
>> Hi Michael,
>>
>> I had tried the syntax you mention:
>>
>> ADD_TEST(NAME "python test"
>>        COMMAND ${PYTHON_EXECUTABLE} $<TARGET_FILE:my_test>)
>>
>> But what happened is that the test gets put in configuration-dependent
>> if statements, e.g. the cTestTestfile.cmake looks similar to:
>>
>> IF("${CTEST_CONFIGURATION_TYPE}" MATCHES "^([Dd][Ee][Bb][Uu][Gg])$")
>>      ADD_TEST(  ... path matching debug ...)
>> ENDIF("${CTEST_CONFIGURATION_TYPE}" MATCHES "^([Dd][Ee][Bb][Uu][Gg])$")
>> IF("${CTEST_CONFIGURATION_TYPE}" MATCHES
>> "^([Rr][Ee][Ll][Ee][Aa][Ss][Ee])$")
>>      ADD_TEST(  ... path matching release ...)
>> ENDIF("${CTEST_CONFIGURATION_TYPE}" MATCHES
>> "^([Rr][Ee][Ll][Ee][Aa][Ss][Ee])$")
>> etc.
>>
>> Because of the if statements, the tests only show up when running
>> ctest with a specific configuration, e.g. "ctest -C Debug" or "ctest
>> -C Release". They do not show up when running just "ctest", even if
>> ctest chooses to run a specific configuration anyway.
>>
>> On the other hand, when using add_test(testname Exename arg1 arg2
>> ...), the CTestTestfile.cmake contains an Exename that includes the
>> basepath. In visual studio usually the basepath actually doesn't
>> contain the executable. When ctest is run though, it actually replaces
>> this basepath with the correct path (often basepath/Debug or
>> basepath/Release).
>>
>> So apparently  when ctest is run without specifying a configuration,
>> it knows where to find the executables that were build with cmake ( it
>> knows to look into the right debug or release sub directory), it  just
>> seems to do this substitution only for the second argument in add_test
>> and i'm wondering if it is possible to get the substitution for other
>> arguments as well, for example by specifying a variable. Or is the
>> recommended way to use the ADD_TEST( NAME... COMMAND ....) syntax and
>> make sure people specify a configuration (e.g. run ctest -C ... )? My
>> only problem with the latter is that if people forget to specify the
>> configuration, not all tests will be run.
>>
>> Thanks,
>>
>> Iman
>>
>>
>>
>> On Sat, Aug 21, 2010 at 1:21 PM, Michael Hertling <mhertling at online.de>
>> wrote:
>> > On 08/20/2010 09:44 PM, Iman Brouwer wrote:
>> >> Hello,
>> >>
>> >> The path to a binary target built by cmake depends on the
>> >> configuration type when using the Visual Studio generators. Is it
>> >> possible to pass this path the ctest without having to run ctest with
>> >> the -c option?
>> >>
>> >> I'd like to do the following:
>> >> I create an executable with cmake as follows, and add a test for it:
>> >>
>> >> add_executable( my_test main.cpp )
>> >> add_test( "test_name" my_test )
>> >>
>> >> Let's assume this executable gets built in bin_path\debug or
>> >> bin_path\release, dependent on the configuration. Cmake generates the
>> >> following line in the CTestTestfile.cmake:
>> >>
>> >> ADD_TEST( test_name "bin_path/my_test" )
>> >>
>> >> Note that the my_test path will not contain the executable, it will be
>> >> in a configuration dependent subdirectory. However, when running
>> >> "ctest", ctest will automatically pick a configuration and will change
>> >> the path in CTestTestfile.cmake to the right path for this
>> >> configuration.
>> >>
>> >> In my case i'd like to call a python script to which i pass the
>> >> executables name including path, so i'd to do something like:
>> >>
>> >> add_test( "python test" ${PYTHON_EXECUTABLE}  my_test )
>> >>
>> >> In this case, "my_test" does not get expanded in CTestTestfile.cmake
>> >> to "bin_path/my_test". Also, if I replace "my_test" here with
>> >> "bin_path/my_test", ctest will not do the automatic substitution to of
>> >> the base path to the right configuration dependent path anymore.
>> >>
>> >> Alternatively, i tried replacing the add_test line with the add_test(
>> >> NAME.... COMMAND ...)  syntax, but then all the tests are inside ctest
>> >> configuration specific ifs. Basically the tests drops of the list of
>> >> tests when just running "ctest", it only appears when running ctest
>> >> with an explicit configuration, e.g. "ctest -C Debug".
>> >>
>> >> Is it possible to have ctest automatically figure out the right path
>> >> to a cmake-built binary, without having to run ctest with an explicit
>> >> configuration?
>> >
>> > AFAIK, the "generator expressions" have been added to ADD_TEST() in
>> > order to address this issue; have you already tried the following?
>> >
>> > ADD_TEST(NAME "python test"
>> >         COMMAND ${PYTHON_EXECUTABLE} $<TARGET_FILE:my_test>)
>> >
>> > Regards,
>> >
>> > Michael
>> > _______________________________________________
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html
>> >
>> > Please keep messages on-topic and check the CMake FAQ at:
>> > http://www.cmake.org/Wiki/CMake_FAQ
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://www.cmake.org/mailman/listinfo/cmake
>> >
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.cmake.org/mailman/listinfo/cmake
>
>