[CMake] Make Timeout not be a failure

[Kyle Edwards]

On Thu, 2019-06-20 at 15:45 -0400, Donald MacQueen [|] via CMake wrote:
> I have a test where I start a program that I know will create some 
> output that I can test.
> 
> But I have no way to kill this program, so I let Ctest kill it with
> a 
> TIMEOUT.
> 
> The next step greps the output to see if it worked.
> 
> So it would be nice if  PASS_REGULAR_EXPRESSION  could override
> TIMEOUT 
> and not mark the test as failed.
> 
> For example:
> 
>      set_tests_properties(${importMaps} PROPERTIES 
> PASS_REGULAR_EXPRESSION "some string I know I will find")
>      #                                                         this 
> ^^^^^^^^^^^^^^^^^^^^^^ makes the test pass even if it times out
>      set_tests_properties(${importMaps} PROPERTIES TIMEOUT 60)

You could wrap your test in a CMake script that calls execute_process()
with a TIMEOUT argument, and then greps the output of the command for
the desired expression. For example:

CMakeLists.txt:

add_test(NAME mytest COMMAND ${CMAKE_COMMAND} -DMYEXE=${PATH_TO_MYEXE}
-P ${CMAKE_CURRENT_LIST_DIR}/ExecuteTest.cmake)

ExecuteTest.cmake:

execute_process(COMMAND ${PATH_TO_MYEXE} TIMEOUT 60 OUTPUT_VARIABLE
output)
if(NOT output MATCHES "^my_desired_regex$")
  message(FATAL_ERROR "myexe did not produce desired output")
endif()

Hope that helps

Kyle