[CMake] Adding compile and build type tests to CMake/CTest

[Edward Diener]

On 9/7/2017 3:32 AM, CHEVRIER, Marc wrote:
> Rather than building a STATIC library, you can build an OBJECT library. In this case only compilation step is done.

So your are saying that:

add_library(foo_compile_test OBJECT EXCLUDE_FROM_ALL foo_compile_test.cpp)
add_test(NAME foo_compile_test
      COMMAND ${CMAKE_COMMAND} --build . --target foo_compile_test 
--config $<CONFIGURATION>
      WORKING_DIRECTORY ${CMAKE_BINARY_DIR})

should work properly ?

> 
> On 07/09/2017, 08:59, "CMake on behalf of Edward Diener" <cmake-bounces at cmake.org on behalf of eldlistmailingz-5p0dqD/c5LGWd6l5hS35sQ at public.gmane.org> wrote:
> 
>      On 9/6/2017 11:39 PM, P F via CMake wrote:
>      > The `add_test` function can run whatever command you want it to, including compiling a target:
>      >
>      > add_library(foo_compile_test STATIC EXCLUDE_FROM_ALL foo_compile_test.cpp)
>      > add_test(NAME foo_compile_test
>      >      COMMAND ${CMAKE_COMMAND} --build . --target foo_compile_test --config $<CONFIGURATION>
>      >      WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
>      >
>      > Then you set cmake to expect failure:
>      >
>      > set_tests_properties(foo_compile_test PROPERTIES WILL_FAIL TRUE)
>      >
>      > You can also check for output instead of just expecting failure:
>      >
>      > set_tests_properties(mytest PROPERTIES
>      >    PASS_REGULAR_EXPRESSION "foo failed"
>      > )
>      >
>      > This is especially useful to make sure that the error is from the static_assert message and not from another compile error.
>      >
>      > Of course, cmake could provide a module which provides a function to do this, which additionally could help workaround the caveats of this approach. This is essentially what `bcm_test` in the boost cmake modules do. > > Hopefully, in the future this module could be merged upstream into cmake.
>      
>      It seems like a hack to have to build a CMake target, with the
>      add_library call, just to compile a source file as a test. You end up
>      creating libraries, even if they are static libraries, just to test a
>      compile. I understand that it might be seen that creating static
>      libraries is no different than creating object files, but this technique
>      appears to me a limitation of CMake. I should be able to add a test
>      which just attempts to compile source file(s) into object file(s), not
>      create static libraries.
>      
>      >
>      >
>      >> On Sep 5, 2017, at 11:44 AM, Edward Diener <eldlistmailingz at tropicsoft.com> wrote:
>      >>
>      >> On 9/5/2017 2:47 AM, Dvir Yitzchaki wrote:
>      >>> There's already CheckCXXSourceCompiles and friends.
>      >>> The only problem is that try_compile is not scriptable otherwise you could let the test invoke
>      >>> ${CMAKE_COMMAND} -P check_source_compiles.cmake.
>      >>
>      >> To put it succinctly CMake should adding compile-time testing so that when some compilation succeeds the test is successful and if the compilation fails the test is not successful, with the proviso that you can reverse the result as a compile should fail type of test. Similarly a build type testing, without having to run anything should be added along the same lines.
>      >>
>      >> In modern C++ it is perfectly feasible, especially with template programming, to do compile time testing, invoking a compile-time static assert as a compile-time failure. Boost has had this for years and modern C++ has it as part of the latest version of the C++ standard. CMake needs to update itself to the reality that pure compile-time testing is a reality for modern C++ and should update itself accordingly. Only having run-time testing is an artifact of the past. Hopefully CMake developers will get the message and make the necessary update to CMake/CTest.
>      >>
>      >>> -----Original Message-----
>      >>> From: CMake [mailto:cmake-bounces at cmake.org] On Behalf Of Roger Leigh
>      >>> Sent: Monday, September 4, 2017 16:51
>      >>> To: cmake at cmake.org
>      >>> Subject: Re: [CMake] Adding compile and build type tests to CMake/CTest
>      >>> On 04/09/17 14:40, Edward Diener wrote:
>      >>>> Boost Build has tests for running an application successfully or not,
>      >>>> for compiling one or more source files successfully or not, and for
>      >>>> building one or more source files into an exe or not. These tests in
>      >>>> Boost Build are the run/run-fail, compile/compile-fail, and
>      >>>> link/link-fail rules.
>      >>>>
>      >>>> CMake/CTest has the exact equivalent to the run/run-fail rule in its
>      >>>> add_test framework, but there is not add_test equivalent to the other
>      >>>> two sets of rules. It sure would be nice, when Boost transitions to
>      >>>> using CMake/CTest instead of Boost Build, if CMake/CTest had the
>      >>>> equivalent of the other two sets of types of test in its add_test
>      >>>> framework.
>      >>>>
>      >>>> Is there any consensus that these other two types of tests might be
>      >>>> valuable for CMake/CTest, or any way to make this happen ?
>      >>> I've certainly wished for them.  Particularly when testing templated code where you want to test that certain things fail correctly, e.g. via static_assert or simply being invalid.
>      >>> I understand it's possible to make this work partially, by creating targets which aren't built by default, and then add tests which invoke the targets.  But this appears to have some caveats, such as potential misbehaviour with parallel testing.  Or you can have a separate CMake build for each individual target, but it's extra complexity.  Having a proper means of registering such tests would be very, very nice.
>      >>> Regards,
>      >>> Roger