CMake/Testing With CTest
Simple Testing
CMake has support for adding tests in the project. To do that, insert the following command in the CMakeLists.txt file:
ENABLE_TESTING()
From that point on, you can add tests in the project using ADD_TEST command:
ADD_TEST(SimpleTest ${EXECUTABLE_OUTPUT_PATH}/SimpleTest Hello)
After building the project, you should be able to do:
1. Unix
make test
2. On GUI development environments such as Visual Studio rebuild the target RUN_TESTS
For more information about ENABLE_TESTING and ADD_TEST, Look at CMake Documentation or run:
cmake --help cmake --help-full cmake --help-command-list
or
cmake --help-command ENABLE_TESTING cmake --help-command ADD_TEST
Dashboards
Testing Dashboards are web pages that display overview of the project testing. The testing clients configure and build the project, as well as run some number of tests. The results of these operations are then submitted to the central server, which prepares the overview pages. Examples of Testing Dashboards are: VTK, ITK, and ParaView.
To enable submission to Dart Testing Dashboard, include the following in CMakeLists.txt:
INCLUDE(Dart)
By default the settings will be to submit to Kitware's Public Dashboard. In order to submit to some other dashboard, create file "DartConfig.cmake" in the toplevel source directory and set the dashboard preferences.
Example of this file is:
# Dashboard is opened for submissions for a 24 hour period starting at
# the specified NIGHLY_START_TIME. Time is specified in 24 hour format.
SET (NIGHTLY_START_TIME "23:00:00 EDT")
# Dart server to submit results (used by client)
IF(DROP_METHOD MATCHES http)
SET (DROP_SITE "public.kitware.com")
SET (DROP_LOCATION "/cgi-bin/HTTPUploadDartFile.cgi")
ELSE(DROP_METHOD MATCHES http)
SET (DROP_SITE "public.kitware.com")
SET (DROP_LOCATION "/incoming")
SET (DROP_SITE_USER "ftpuser")
SET (DROP_SITE_PASSWORD "public")
ENDIF(DROP_METHOD MATCHES http)
SET (TRIGGER_SITE
"http://${DROP_SITE}/cgi-bin/Submit-vtk-TestingResults.pl")
This will submit testing results to the VTK dashboard.
CTest - Client for Dart Dashboard Server
CTest is a testing program distributed as a part of CMake. It can perform simple task of running a set of tests but it can also generate and submit Dart compatible Dashboard results. The good thing about CTest is that if you use CMake, you already have CTest. Detailed description of CTest option can be seen by running:
ctest --help
or
ctest --help-full
To convert existing Dart Client run from the project, find lines like:
cd ProjectNightlyBuildDirectory tclsh /location/of/Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl \ Nightly Start Update Configure Build Test Submit
and convert them to CTest style:
cd ProjectNightlyBuildDirectory ctest -D Nightly
Dashboard can be also generated in stages. This way partial testing results can be submitted and seen before long operations are completed:
cd ProjectNightlyBuildDirectory ctest -D NightlyStart ctest -D NightlyUpdate ctest -D NightlyConfigure ctest -D NightlyBuild ctest -D NightlySubmit ctest -D NightlyTest ctest -D NightlyCoverage ctest -D NightlySubmit ctest -D NightlyMemCheck ctest -D NightlySubmit
Finally CTest extends functionality of Dart by adding:
- HTTP submission support
- Customization of the testing by providing:
- Custom build error/warning regular expressions
- Ability to suppress some tests from being tested or memory checked and ability to run subset of tests
- Ability to run commands before and after tests are run
- Ability to run whole testing process described in a single script
For an example of how CTest can run the whole testing process described in a single script, look at how CMake dashboards are created with the ctest -S script.
More information about CTest can be found in Mastering CMake.
