[CMake] Custom target for running tests as part of the build

[Matthew Keeler]

One thing I see immediately is this line:

$<TARGET_FILE:${TEST_NAME}> > ${TEST_NAME}.output || cat ${TEST_NAME}.output

Unless ${TEST_NAME}.output doesn’t exist cat is going to have a successful
exit status and cause the whole COMMAND to be successful. Something like
“|| (cat ${TEST_NAME}.output && false)” should work in bash. When the test
passes everything in the parens won’t be executed but when it does it will
still force a bad exit status.
For a cross platform solution I would create a cmake script to do the grunt
work of executing the test.

COMMAND ${CMAKE_COMMAND} -DTEST_EXEC=<TARGET_FILE:${TEST_NAME}>
-DTEST_NAME=${TEST_NAME} -P test-runner.cmake

Then in the script it could do something like the following:

execute_process(COMMAND ${TEST_EXEC} OUTPUT_FILE ${TEST_NAME}.out
ERROR_FILE ${TEST_NAME}.out RESULT_VARIABLE TEST_SUCCESS)
if (NOT TEST_SUCCESS EQUAL 0)
    execute_process(COMMAND cat ${TEST_NAME}.out)
    message(FATAL_ERROR “Execution of ${TEST_EXEC} exited with a failure
status”)
endif()

execute_process(COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed)


Depending on how much you need the cross platform solution the ugliness of
the extra indirection may not be desirable for you.

-- 
Matt Keeler


On August 23, 2016 at 11:20:48, Steve Lorimer (steve.lorimer at gmail.com)
wrote:

We have several unit tests which we would like to be run as part of our
build process.

To achieve this I have a helper script which creates a custom command that
runs the test, and if successful, creates a file "test_name.passed".

I then add a custom target "test_name.run" which depends on
"test_name.passed".

The idea is that if "test_name.passed" doesn't exist or is older than
"test_name", the custom command will be run.

Builds will continue to run the custom command until the test passes.
Subsequent builds won't call the custom command, so the test won't be run
if it doesn't need to.

Here is the script:

    # create test.passed command which runs the test and creates a sentinel
file if it passes
    add_custom_command(
        OUTPUT  ${TEST_NAME}.passed
        COMMAND $<TARGET_FILE:${TEST_NAME}>
        COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
        DEPENDS ${TEST_NAME}
        )

    # create test.run module which depends on test.passed
    add_custom_target(${TEST_NAME}.run
        ALL
        DEPENDS ${TEST_NAME}.passed
        )

The problem is that our tests often log a tonne of information to stdout,
and it makes for a very noisy build.

I'm trying to now capture stdout to a file, and only in the event of a
failure, display the test output.

My first attempt was to try a Bash shell scripting syntax - capture stdout
into a file and when the exit status is an error, cat the file.

    add_custom_command(
        OUTPUT  ${TEST_NAME}.passed
        COMMAND $<TARGET_FILE:${TEST_NAME}> > ${TEST_NAME}.output || cat
${TEST_NAME}.output
        COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
        DEPENDS ${TEST_NAME}
        )

This doesn't work, as even if the test fails I am getting the sentinal
"test_name.passed" file created, which means the next time I try to build
it thinks the test passed.

Is there a way to achieve what I want? Bonus points for a cross-platform
method, but if it has to be Linux only, then so be it.

Thanks in advance
Steve

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20160823/754ccf77/attachment.html>