CDash:Testing/Nightly

The purpose of this document is to help you setup a simple script that can be called by cron or Windows Task Scheduler to perform a nightly test of CDash. First I will lay out the issues that keep this from being completely straightforward. Then I will present example scripts.

Issues

Note that this list of issues is very Linux-centric. Many or all of these issues may not apply if you're setting up nightly CDash testing on Windows.

Permissions for the xdebugCoverage directory

Under Linux, the web server is typically run by its own user. This is a problem when testing CDash because the build directory is deleted and recreated with permissions of the dashboard user each night. If we're not careful the webserver will be unable to write coverage files to the xdebugCoverage directory.

How do we overcome this issue? One simple way is to run the test as the webserver user.

Another more complicated, but potentially more secure method is to use the set-groupID bit and umask. Note that this only works if your build directory is a subdirectory of your source directory.

First go to your checkout of CDash and run:

 sudo chgrp www-data .

This assumes that your webserver user is named "www-data". Change it to whatever is appropriate otherwise.

Next set the groupID bit on your CDash source dir:

 chmod g+s .

Now if we execute the following command before running ctest, our webserver user will have no problems writing out coverage files.

umask 2

Having a display available for Selenium testing

Selenium based tests will fail if they run while your screen is locked. We can work around this issue with VNC. First install a VNC server & (optional) viewer on your system. On Ubuntu, you can do so with the following command:

sudo apt-get install vnc4server xvnc4viewer

You'll need to make sure the VNC server is running before the nightly CDash tests begin.

One option is to just leave your VNC server running all the time. You can do this by reconfiguring Ubuntu to run your VNC server whenever your computer boots up. This can be set under System -> Preferences -> Startup Applications .

Another option is to start the VNC server right before you call CTest. See the example below on starting & stopping the Selenium server for an example on how to do something like this.

Once your VNC server is running you'll need to export the appropriate display in your dashboard script. See the example shell script below for more details.

Starting and stopping the Selenium server

The Selenium-based tests in CDash won't be able to execute properly if the Selenium-RC server isn't running. Therefore my dashboard script starts the Selenium Server before running CTest, and then kills it when the nightly tests are complete.

Examples

These scripts show how to schedule a nightly build with all options turned on.

shell script

This is what should be called from cron each night <source lang="bash">

1. !/bin/sh

export DISPLAY=localhost:2 umask 2 /usr/bin/java -jar /path/to/selenium-server-1.0.3/selenium-server.jar & seleniumPID=$! /usr/local/bin/ctest -VV -S /path/to/my_cdash_script.cmake > /path/to/Logs/cdash_nightly.log 2>&1 kill $seleniumPID </source>

cmake script

This is invoked by the shell script above <source lang="cmake">

1. This script requires 2.8.1.20100527 or later.
2. It needs Bill Hoffman's php/xdebug coverage analysis
3. code recently added to ctest.

set(CTEST_SITE "myhost.mydomain") set(CTEST_BUILD_NAME "mybuildname-coverage") set(CTEST_CMAKE_GENERATOR "Unix Makefiles") set(CTEST_UPDATE_COMMAND "svn") set(CTEST_SOURCE_DIRECTORY "/path/to/CDashTesting") set(CTEST_BINARY_DIRECTORY "/path/to/CDashTesting/build")

ctest_empty_binary_directory(${CTEST_BINARY_DIRECTORY})

ctest_start(Nightly) ctest_update()

set(cfg_options

 -DPHP_EXE=/usr/bin/php
 -DPHPUNIT_EXE=/usr/bin/phpunit
 -DCDASH_SITE=${CTEST_SITE}
 -DCDASH_DB_LOGIN=root
 -DCDASH_DB_PASS=
 -DCDASH_DB_HOST=localhost
 -DCDASH_DB_TYPE=mysql
 -DCDASH_USE_SELENIUM=ON
 -DCDASH_USE_APACHE2=ON
 -DCLIENT_TEST_SCRIPT1=/path/to/client-part1.ctest
 -DCLIENT_TEST_SCRIPT2=/path/to/client-part2.ctest
 -DCMake_SOURCE_DIR=/path/to/CMake-src
 )

ctest_configure(OPTIONS "${cfg_options}")

ctest_read_custom_files(${CTEST_BINARY_DIRECTORY}) ctest_test() ctest_coverage() ctest_submit() </source>