CDash:Testing: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
No edit summary
Line 1: Line 1:
CDash integrates a self-testing framework based on [http://www.simpletest.org Simple Test]. The self testing code is located in the ''CDash/testing'' directory.
CDash integrates a self-testing framework based on [http://www.simpletest.org Simple Test]. The self testing code is located in the ''CDash/testing'' directory.


= Submitting a CDash Dashboard to the CDash dashboard =
CDash testing requires the use of CMake and CTest.  You can download the CMake suite of tools from [http://www.cmake.org/cmake/resources/software.html www.cmake.org].  CDash testing works much the same as any other CMake based project.


Don't worry, it's not a mistake in the title of this section, self-testing involves submitting a CDash submission to CDash.
= Checkout the CDash source code =


=== Testing CDash with CTest ===
See [[CDash:Installation#Downloading_CDash|Downloading CDash]] for instructions on obtaining a copy of the CDash source code from subversion.  During this step you should decide whether you want to test the latest release of CDash or the current development version.
CDash can be tested with CTest from cmake (www.cmake.org.).  For this style of testing, you treat CDash much like any other CMake based project.


* Checkout the CDash source code from the svn repository to a directory named "CDashTesting" in your web server's htdocs folder.
No matter which version of CDash you choose to test, make sure to check out the source code into your web server's htdocs folder.


* assume that the full path to CDash is /path/to/source/CDashTesting
= Setup CDash for testing =


* Create a build directory
== Create a build directory ==
  cd /path/to/source/CDashTesting
  mkdir build


* Run cmake on on CDashTesting
cd /path/to/CDashTesting
  cd build
mkdir build
  cmake ..


  There are some CMake cache options that configure server names, the default is localhost.
==Run CMake from the build directory ==


* For coverage, you need to have xdebug installed as a module in your php installation. Once that is done, you need to edit the php.ini file, and add these two lines and restart apache:
cd build
  auto_prepend_file =/path/to/source/CDashTesting/prepend_coverage.php
cmake ..
  auto_append_file =/path/to/source/CDashTesting/build/append_coverage.php


**For coverage to work, a CMake next newer than 5/25/2010 at 4:00 pm will be needed.
== Define CMake variables ==
**Using XAMPP 1.7 on Windows Vista/7 You might need a different version of XDebug: http://xdebug.org/files/php_xdebug-2.1.0RC1-5.3-vc6.dll. The version shipped with XAMPP causes the server to crash: http://docs.joomla.org/Setting_up_your_workstation_for_Joomla!_development#Edit_PHP.INI_File


* Note this will create '''CDashTesting/cdash/config.local.php''' and '''CDashTesting/testing/config.test.local.php''' files in your source tree, so svn will show them as ? files.
CDASH_DB_HOST
CDASH_DB_LOGIN
CDASH_DB_PASS
CDASH_DB_TYPE
*These four parameters tell CDash how to connect to your local database server.


* Run ctest on the build tree, like any other ctest project. For example, this will run an experimental coverage build:
CDASH_SERVER
  cd /path/to/source/CDashTesting/build
*CDASH_SERVER should indicate how to connect to your local system via a web browser.  This is typically the system's host name, or simply localhost.
  ctest -D ExperimentalStart -D ExperimentalConfigure -D ExperimentalTest -D ExperimentalCoverage -D ExperimentalSubmit


=== Configuring the testing ===
CDASH_USE_APACHE2
Here are the steps on how to configure the testing without ctest:
*Setting this to ON will enable tests specific to the Apache web server.


* Checkout the CDash source code from the svn repository to a directory named "CDashTesting" in your web server's htdocs folder.
CDASH_USE_SELENIUM
PHPUNIT_EXE
*Setting these options will enable testing of javascript based functionality of CDash.  Requires [http://seleniumhq.org/ selenium] and [http://www.phpunit.de/ PHPUnit].  More on this below.


* Add a file named '''CDashTesting/cdash/config.local.php''' containing:
CMake_SOURCE_DIR
*Point this to a local checkout of CMake source code if you'd like to import some tests from CMake into CDash.  This will cause CDash to perform a more thorough end-to-end test of the CMake/CTest/CDash suite of tools.


<pre>
PHP_EXE
<?php
*CDash testing requires a command-line version of PHP to be installed on the system.
$CDASH_DB_NAME = 'cdash4simpletest';
$CDASH_TESTING_MODE = true;
?>
</pre>


This file is automatically included by CDash's '''config.php''', but only if it exists.
= Test CDash =


The database has to be named exactly 'cdash4simpletest' for the tests to run. Part of the test suite involves destroying the existing database of that name and re-creating it from scratch. Because the testing of CDash drops and creates the database you need to make sure this database is not being used for any other purpose than self testing. Also, make sure that the $CDASH_DB_LOGIN has enough privileges to create/drop databases on your SQL server.
This is as simple as entering your CDash build directory and running CTest
  cd /path/to/CDashTesting/build
  ctest -VV


* Add a file named '''CDashTesting/testing/config.test.local.php''' containing (example values only... adjust to suit your localhost web server):
Note this will create '''CDashTesting/cdash/config.local.php''' and '''CDashTesting/testing/config.test.local.php''' files in your source tree, so svn will show them as ? files.


<pre>
= Advanced Topics =
<?php
$configure = array(
  'urlwebsite'      => 'http://localhost/CDashTesting',
  'outputdirectory'  => 'C:/tmp',  // or /tmp or ~/tmp on a Linux/Mac server
  'type'            => 'Nightly',
  'site'            => 'computername.companyname',
  'buildname'        => 'SVN-Win32-xampp',
  'cdash'            => 'http://www.cdash.org/CDash',
  'svnroot'          => 'C:/xampp/htdocs/CDashTesting'
  );
?>
</pre>


This file is automatically included by CDash's '''config.test.php''', but only if it exists.
[[CDash:Testing/Coverage|Code coverage]]


* '''urlwebsite''' is the URL to access the tested installation of CDash. Since simpletest emulates a web browser you need to make sure that the urlwebsite is accessible locally. That is, you should be able to visit this URL in a web browser on the testing machine and see the main CDash page.
[[CDash:Testing/Selenium|Testing javascript functionality with Selenium]]
* '''outputdirectory''' is a directory where the testing output can be stored on the system. Make sure that the current user can write to that directory.
* '''type''' is the type of submission for CDash: Nightly, Continuous or Experimental.
* '''site''' is the name of the site submitting to CDash.
* '''buildname''' is name of the build.
* '''cdash''' is the URL to the current dashboard. Note that the project should be named CDash, i.e. http://www.cdash.org/CDash/submit.php?project=CDash
* '''svnroot''' is the file system path to your testing installation of CDash. The self testing does an svn update to check what files have changed.


=== Starting the testing ===
[[CDash:Testing/Client Management|Client management]]
Then to launch the tests and submit a dashboard:
  cd CDashTesting/testing
  php5 alltests.php


For xampp installations on Windows, run:
[[CDash:Testing/Nightly|Setting up nightly Builds]]
  cd CDashTesting\testing
  C:\xampp\php\php.exe alltests.php
 
To run the tests in your browser (without submitting a dashboard) visit the URL:
  http://localhost/CDashTesting/testing/alltests.php

Revision as of 18:27, 25 June 2010

CDash integrates a self-testing framework based on Simple Test. The self testing code is located in the CDash/testing directory.

CDash testing requires the use of CMake and CTest. You can download the CMake suite of tools from www.cmake.org. CDash testing works much the same as any other CMake based project.

Checkout the CDash source code

See Downloading CDash for instructions on obtaining a copy of the CDash source code from subversion. During this step you should decide whether you want to test the latest release of CDash or the current development version.

No matter which version of CDash you choose to test, make sure to check out the source code into your web server's htdocs folder.

Setup CDash for testing

Create a build directory

cd /path/to/CDashTesting
mkdir build

Run CMake from the build directory

cd build
cmake ..

Define CMake variables

CDASH_DB_HOST
CDASH_DB_LOGIN
CDASH_DB_PASS
CDASH_DB_TYPE
  • These four parameters tell CDash how to connect to your local database server.
CDASH_SERVER
  • CDASH_SERVER should indicate how to connect to your local system via a web browser. This is typically the system's host name, or simply localhost.
CDASH_USE_APACHE2
  • Setting this to ON will enable tests specific to the Apache web server.
CDASH_USE_SELENIUM
PHPUNIT_EXE
  • Setting these options will enable testing of javascript based functionality of CDash. Requires selenium and PHPUnit. More on this below.
CMake_SOURCE_DIR
  • Point this to a local checkout of CMake source code if you'd like to import some tests from CMake into CDash. This will cause CDash to perform a more thorough end-to-end test of the CMake/CTest/CDash suite of tools.
PHP_EXE
  • CDash testing requires a command-line version of PHP to be installed on the system.

Test CDash

This is as simple as entering your CDash build directory and running CTest 

 cd /path/to/CDashTesting/build
 ctest -VV

Note this will create CDashTesting/cdash/config.local.php and CDashTesting/testing/config.test.local.php files in your source tree, so svn will show them as ? files.

Advanced Topics

Code coverage

Testing javascript functionality with Selenium

Client management

Setting up nightly Builds

Retrieved from "https://public.kitware.com/Wiki/index.php?title=CDash:Testing&oldid=24592"