[CMake] Adding compile and build type tests to CMake/CTest

[Edward Diener]

On 9/4/2017 9:51 AM, Roger Leigh wrote:
> On 04/09/17 14:40, Edward Diener wrote:
>> Boost Build has tests for running an application successfully or not, 
>> for compiling one or more source files successfully or not, and for 
>> building one or more source files into an exe or not. These tests in 
>> Boost Build are the run/run-fail, compile/compile-fail, and 
>> link/link-fail rules.
>>
>> CMake/CTest has the exact equivalent to the run/run-fail rule in its 
>> add_test framework, but there is not add_test equivalent to the other 
>> two sets of rules. It sure would be nice, when Boost transitions to 
>> using CMake/CTest instead of Boost Build, if CMake/CTest had the 
>> equivalent of the other two sets of types of test in its add_test 
>> framework.
>>
>> Is there any consensus that these other two types of tests might be 
>> valuable for CMake/CTest, or any way to make this happen ?
> 
> I've certainly wished for them.  Particularly when testing templated 
> code where you want to test that certain things fail correctly, e.g. via 
> static_assert or simply being invalid.

I think that CMake has missed out on the simple fact that in modern C++ 
you can create compile-time tests/asserts rather than rely on run-time 
tests. In other words CMake has no provision for compile-time testing, 
only run-time testing. In this respect CMake is definitely behind the 
curve of modern C++ development. Of course unless the CMake developers 
wake up to this simple fact, and plan to provide compile-time testing, 
and possibly even build testing, ala Boost Build, to CMake it will 
remain behind the curve for modern C++ development.

When the Boost Steering Committee decided to move Boost to using CMake 
in the near future, it may not have taken into account the fact that 
CMake might be missing something that modern C++ development in Boost 
uses via Boost Build. Unless there is some sort of push back to the 
CMake developers so that the needed functionality is added to CMake, it 
is going to be a long, hard road for converting from Boost Build to 
CMake for the Boost library and tools. This is not a knock against 
CMake, which is widely used and widely popular, but rather a recognition 
that Boost uses modern C++ and CMake may not provide all the 
building/testing facilities needed for a modern C++ development 
environment such as Boost.

> 
> I understand it's possible to make this work partially, by creating 
> targets which aren't built by default, and then add tests which invoke 
> the targets.  But this appears to have some caveats, such as potential 
> misbehaviour with parallel testing.  Or you can have a separate CMake 
> build for each individual target, but it's extra complexity.  Having a 
> proper means of registering such tests would be very, very nice.
> 
> 
> Regards,
> Roger