[CMake] CTest + Catch Framework

[Robert Dailey]

I wouldn't say this is an artificial problem. let me go into more detail.

A single test can define any number of test cases. These are unknown
to CMake scripts, as they are written and defined in code. The ideal
solution is to have the test program output a list of all test cases
and based on that list, automatically generate the add_test commands.
That way I don't have to worry about hard-coding test case names in
the build scripts. When a new test case is added to the test code (no
new files would be added), "magically" a new unit test appears to
CMake.

But none of this can be done unless the unit test executables are
built. Thus the chicken and egg problem. To do it as you suggest would
require me to hard-code the test cases. Example:

In MyTest.cpp:

#include <catch.hpp>
TEST_CASE("test1") {}
TEST_CASE("test2") {}

In CMakeLists.txt:

add_executable( MyTest MyTest.cpp )
add_test( NAME MyTest_test1 COMMAND test1 )
add_test( NAME MyTest_test2 COMMAND test2 )

Am I correctly understanding what you have in mind? If so, can you
think of a way to automate this a little better?

On Tue, Jan 27, 2015 at 7:18 AM, David Cole <DLRdave at aol.com> wrote:
> It's only a chicken and egg problem if you constrain yourself artificially.
>
> When you define a new unit test source file, there are two things you
> need to do:
> (1) add it to its executable
> (2) write an add_test line that invokes that specific unit test
>
> You could possibly encapsulate those actions into an add_unit_test
> macro so that it only seems like you have one thing to do, but
> certainly, it's possible.
>
> You must be doing #1 already, so you just have to make sure #2 always
> happens whenever #1 happens.
>
> Both eggs.
>
>
> D
>
>
>
>
> On Mon, Jan 26, 2015 at 4:05 PM, Robert Dailey <rcdailey.lists at gmail.com> wrote:
>> I believe so, but then you run into the chicken and egg problem:
>>
>> I need to fully build all unit tests in order to know all the test
>> cases, but I need to define the tests at the generation phase (before
>> build).
>>
>> I'm just not sure how to handle this. The best i can think of is to
>> generate 1 test executable per CPP file that we write in our unit
>> tests (generally 1 CPP file per class that is unit tested). Any ideas?
>>
>> On Mon, Jan 26, 2015 at 10:05 AM, David Cole <DLRdave at aol.com> wrote:
>>> Can you run a command that executes one test case at a time, or do you
>>> have to run all 50 when you execute the tests for a library?
>>>
>>> ctest just runs whatever you give it with add_test -- if you want to
>>> filter a collection of unit tests to run only a single unit test, then
>>> the unit test framework you're using would have to support that. Does
>>> Catch allow you to run the tests individually?
>>>
>>>
>>>
>>> On Mon, Jan 26, 2015 at 12:30 AM, Robert Dailey
>>> <rcdailey.lists at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I'm using Catch as my unit test framework:
>>>> https://github.com/philsquared/Catch
>>>>
>>>> Is it possible for CTest to report each TEST_CASE block as an
>>>> individual test? My understanding is that CTest will only treat each
>>>> executable as a test. However, my test structure is as follows:
>>>>
>>>> 1 library
>>>> 1 test project
>>>>
>>>> Each test project has 1 test CPP file per each class in the library.
>>>> This way I implement tests for classes in a corresponding CPP file.
>>>>
>>>> Each CPP file contains multiple test cases (defined by TEST_CASE macro).
>>>>
>>>> The resulting output of `ctest -T Test` shows only 1 test, even though
>>>> that may be around 50 test cases. I'd like CMake to show the pass/fail
>>>> status of each one. Is this possible?
>>>> --
>>>>
>>>> Powered by www.kitware.com
>>>>
>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>
>>>> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>>>>
>>>> CMake Support: http://cmake.org/cmake/help/support.html
>>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>>>
>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://public.kitware.com/mailman/listinfo/cmake