CDash:Documentation: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
 
(75 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[CDash | < CDash Main Page]]
This page is the official CDash documentation.
This page is the official CDash documentation.


Line 14: Line 16:


= Login to CDash =
= Login to CDash =
CDash uses secure login for users and adminsitrators. CDash does not store any plain text passwords in the database, therefore passwords cannot be recovered, even by system administrators. LDAP is not currently supported by CDash.
CDash uses secure login for users and adminsitrators. CDash does not store any plain text passwords in the database, therefore passwords cannot be recovered, even by system administrators.  
 
''Note: LDAP is not currently supported.''


* To login, click on the 'Login' link at the top of the main CDash page or directly go to the user.php page http://myserver/CDash/user.php.
* To login, click on the 'Login' link at the top of the main CDash page or directly go to the user.php page http://myserver/CDash/user.php.
* If you don't have a login, click on the 'register' link on the top menu. In order to register, CDash requires a valid email address, your first name, last name and password. The institution is optional. CDash implements some security features to avoid robots to register.
* If you don't have a login, click on the 'register' link on the top menu. In order to register, CDash requires a valid email address, your first name, last name and password. The institution is optional. CDash implements some security features to avoid robots to register.
* Once register you are ready to type your email and password provided in the login screen.
* Once registered you are ready to type your email and password provided in the login screen.
* The default login/password after installation is 'admin[at]cdash.org' and 'administrator'.
 
= Available Dashboards =
[[Image:CDashIndexTable.jpg|thumb|CDash index table]]
 
The first page display on CDash is a list of current public projects (private projects are not shown on this page). Several information are shown on this page:
 
* The name of the project
* The total number of submissions for each project
* The date of the first build for each project
* The date of the last submission (not build time)
 
* To total size of the database
 
Clicking on the name of the project redirects the user to the main dashboard for this project.
 
= Project Dashboard =
[[Image:CDashMainDashboard.jpg|thumb|Project's dashboard page]]
[[Image:CDashHelp.jpg|thumb|Help link from the main dashboard page]]
 
Each project dashboard have a similar layout:
* Top row of the page:
** Link to "My CDash/Login"
** Current date of the server hosting CDash
** Link to RSS feed to the project (if not private)
* Banner:
** Custom project logo (see [[CDash:Administration#Updating_project.E2.80.99s_logo | Updating project's logo]] to upload a new logo). ''Note: clicking on the logo redirects to the project's home page (if any defined)''
** Project's name
* Main Menu
** '''Dashboard:''' goes back to the list of projects hosted on CDash
*** '''Updates:''' Overview of the updates for this day and this project
*** '''Builds:''' Overview of the builds for this day and this project
*** '''Tests:''' Overview of the test failing for this day and this project
*** '''Map:''' Map of the sites for this project and this day
** '''Calendar:''' display a calendar to check previous dashboards
** '''Previous:''' Go to the previous day
** '''Current:''' Go to the current day
** '''Next:''' Go to the next day
** '''Project:''' Project links
*** '''Home:''' Home URL
*** '''Doxygen:''' Documentation page for this project
*** '''CVS:''' Link to the CVS/SVN viewer for this project
*** '''Bugs:''' Link to the bug tracker for this project
* '''Daily updates:''' Updates on the project's repository.
* '''Help link:''' popup a window with help information
* Build groups: explained in the next section.
* Coverage: explained in the next section.
* Dynamic Analysis: explained in the next section.
 
== Build groups ==
 
Each build submitted by a site corresponds to a row in the main dashboard. By default the build falls in the group defined by the type of submission (i.e Nightly, Experimental or Continuous), however, custom builds can be defined. See [[CDash:Administration#Build_Groups | Build Groups]] administration section for more information on how to create new groups and define rules for builds.
 
* Mouse hover the top row of a build group, shows the quick link bar to quickly jump to a different group.
* Expected builds are shown as a empty row in the dashboard if they have not been submitting.
 
=== Default group ===
[[Image:CDashSorting.jpg|thumb|Muli-columns sorting]]
[[Image:CDashGroup.jpg|thumb|Group description]]
[[Image:CDashShowNote.jpg|thumb|Note for a build]]
 
In this section we explain the layout of default groups (i.e. Nightly, Experimental, Continuous,...). These groups are represented in the light blue sections.
 
'''Clicking on the gray header allow sorting the table by the clicked field. Multisorting is possible by holding the shift key.''', also the current sorting is saved by the user's web browser using cookies, allowing to keep the same sorting while browsing different days for the same project.
 
Clicking on the group name displays the build group description.


= Creating a project =
Each build (row) contains the following information:
Go to the administrative section of MyCDash (login as administrator) and select [Create new project].
* '''Site:''' Name of the site submitting the build. Clicking on the name redirects to site description.
Here’s a description of the fields.
* '''Build name:''' Name of the build. Clicking on the name redirects to the build summary page.
** '''Build note:''' If submitted with CTest and configure with build notes, a small icon shows the build note. Clicking on the icon displays the CTest script used to submit the page.
** '''CTest version:''' If submitted with CTest, a small icon displays the CTest version.
** '''Build info:''' If a build is expected and has not been submitting, or if a build has errors or warnings or tests failing, then a build info icon is present. Clicking on the build info icon trigger an ajax request that displays more information about this build.
** '''Notes:''' If someone submitted a note for this build, mouse hover the note icon will display the current notes.
* '''Update:''' Shows the number of files updated (from the SVN/CVS repository) for this build. Clicking on the number shows the name and information of the updated files.
* '''Cfg:''' Shows the configuration errors. If red, some errors occurred while configuring the project. Clicking on the number displays the current configuration status.
* '''Build:''' Displays the principal build information.
** '''Error:''' Number of build errors.
** '''Warn:''' Number of build warnings.
** '''Min:''' Time (in minutes) it took to build the project.
* '''Test:''' Displays information about the test.
** '''Not Run:''' Number of test not run (because not found for instance).
** '''Fail:''' Number of test failed. Clicking on the number show the name of the failing tests. If time status is enabled for the project, it is reflected here.
** '''Pass:''' Number of test passed. Clicking on the number show the name of the passing tests. If time status is enabled for the project, it is reflected here.
** '''Min:''' Time it took to run all the tests.
* '''Build time:''' Time when the build started expressed in the timezone of the webserver. This is different from the time when the build was submitted.


* Name: name of the project. CDash allows spaces for the name of the project but it is not recommended. If the project’s name contains space make sure you replace the space by the corresponding HTML entity, i.e. %20.
=== Coverage ===
[[Image:CDashCoverage.jpg|thumb|Coverage section of the dashboard]]


* Description: description of the project.
CDash supports gcov and bullseye as coverage tools.
* Home URL: Home url of the project (with or without http://) . This URL is referred in the top menu of the dashboard for this project.
The main dashboard displays the following information:
* CVS/SVN Viewer URL:
* CVS/SVN View Type: Current CDash supports ViewCVS, Trac, Fisheye and CVSTrac as CVS/SVN viewers. Select the appropriate viewer depending on your current configuration.
* CVS/SVN repository: In order to get the daily updates [put screenshot here], CDash should be able to access the current repository. It is recommended to use the anonymous access, for instance :pserver:anoncvs@myproject.org:/cvsroot/MyProject. If the project needs ssh access, make sure that the user running the websever running CDash as the proper ssh keys. CDash doesn’t store any passwords.


* Bug tracker URL: URL of the bug tracker for this project
* '''Site:''' name of the site
* Documentation URL: URL of the documentation for this project
* '''Build name:''' name of the build
* Logo: small logo for this project. It is recommended to upload a transparent gif to blend with CDash’s banner. The height of the image shouldn’t be more than 100 pixels an optimized look. Project logos are stored in the database directly.
* '''Percentage: ''' Overall coverage percentage (for all files in the project).
* Public dashboard: if the box is checked that means that the dashboard is public and anybody can access the dashboard, claim sites and look at the current status. By default dashboards are private.
* '''Passed:''' Number of lines of code that are tested.
* Coverage threshold: CDash marks the coverage has passed (green) if the global coverage for a build or specific files is above this threshold. It is recommended to set the coverage threshold to a high value and decrease it when the coverage is getting higher.
* '''Failed:''' Number of lines of code that are not tested.
* Nightly start time: CDash displays the current dashboard using a 24hours window. The nightly start time defines the beginning of this window. Note that the start time is expressed in the form HH:MM:SS TZ, i.e. 21:00:00 EDT.
* '''Date:''' Build starting date.
* Google Analytics Tracker: CDash supports visitor tracking through Google analytics. See “Adding Google Analytics” for more information.
* Enable test timing:  Enable/Disable test timing for this project
For more information about test timing see the “CDash Test Timing” section.
* Test time standard deviation:  
Set a multiplier for the standard deviation for a test time. If the time for a test is higher than mean+multiplier*standarddeviation, the test time status is marked as failed. Default is 4 if not specified. Note that changing this value doesn’t affect previous builds but only builds submitted after the modification.
For more information about test timing see the “CDash Test Timing” section.
* Test time standard deviation threshold:
Set a minimum standard deviation for a test time. If the current standard deviation for a test is lower than this threshold then the threshold is used instead. This is particularly important, for tests that have a very low standard deviation but still some variability. Default threshold is set to 2 if not specified. Note that changing this value doesn’t affect previous builds but only builds submitted after the modification.
For more information about test timing see the “CDash Test Timing” section.
* Email broken submission: Enable/Disable sending email for broken submissions for this project. This is a general feature.
* Email build missing: Enable/Disable sending email when a build has not submitted. This feature is currently not implemented.
* Email low coverage: Enable/Disable sending email when the coverage for files is lower than the default threshold value specified above.
* Email  test timing change: Enable/Disable sending email when a test timing has changed. This feature is currently not implemented.
* Download CTest config: Automatically generated CTest configuration file. downloading this file and putting it at the root of your project, allows to quickly get started with CTest/CDash and submitting to the dashboard.


= Editing a project =
=== Dynamic Analysis ===
To edit the current configuration for a project. Log in as administrator and select [edit project] in the administration section. If you have more than one project hosted on CDash, select the project you want to modify.
[[Image:CDashDynamicAnalysis.jpg|thumb|Dynamic Analysys section of the dashboard]]
== Update the project’s logo ==
In order to update the current project’s logo, just select a new image file and click “Update Project”.


See the section “Creating a project” for more information.
CDash supports [http://valgrind.org valgrind] as the main dynamic analysis tool.
The main dashboard displays the following information:


= Test Timing =
* '''Site:''' Name of the site
Added since CDash 1.0. CDash supports timing defects for tests. CDash keeps in the database a current weighted average of the mean and standard deviation for each test time for each build. In order to keep the computation as light as a process as possible, the following formula is used, involving only the previous build.
* '''Build name:''' Name of the build
* '''Checker: ''' Name of the checker.
* '''Defect count:''' Number of memory defects.
* '''Date:''' Build starting date.


  newMean = (1-alpha)*oldMean + alpha*currentTime
== Daily updates ==
  newSD = sqrt((1-alpha)*SD*SD + alpha*(currentTime-newMean)* (currentTime-newMean)


A test is defined as failing if it verifies the following:
CDash stores the daily updates in the database for a quick query. The number of files changed for that day as well as the number of authors who changed the files are displayed. Clicking on the number of files changed display the name of the files and more information.


  if previousSD < thresholdSD then previousSD = thresholdSD.
See the [[CDash:Design#Daily_Updates | Daily Updates Design]] section for more information.
  if  currentTime > previousMean*multiplier*previousSD.


One can notice that alpha defines the current “window” for the computation. By default alpha is set to 0.3.
== Map ==
[[Image:CDashMap.jpg|thumb|Map of the sites contributing to a project]]


== Recompute test timing ==
CDash stores the geolocation from the IP address for the sites and displays them on a map via Google map API. This allows to quickly browse where the builds are coming from. Clicking on a pin on the map, shows the name of the site.
If you just upgraded CDash, you might notice that the current submissions are showing a high number of failing test due to time defects. This is due to the fact that CDash doesn’t have enough sample points to compute the mean and standard deviation for each test, and particularly the standard deviation might be very small (actually zero for the first samples).
It is recommended to turn the “enable test timing” to off  (see edit project section)  for about a week (or until you get enough build submissions and CDash has an approximated mean and standard deviation for each test time.


The other option is to force CDash to compute the mean and standard deviation for the test for the past X days. Be warned that this process might take a long time depending on the number of test and projects involved. In order to recomputed the test timing, go to the [backward compatibility tools] in the administration section and specify the number of days (default is 4) and click on “Compute test timing”. When the process is done the new mean, standard deviation and status should be updated for the tests submitted during this period.
See [[CDash:Design#Map_and_Geolocation | Map and Geolocation]] design page for more information.


= Adding Google Analytics =
= My CDash =
In order to get a Google analytics account:
# First go to http://www.google.com/analytics/indexu.html
# If you don't have an account, set one up.
# Then, add a website profile.
# On CDash, login and edit your project
# Add the code you get from google into the Google Analytics Tracker (i.e. UA-43XXXX-X) for your project.


=  Project Roles =
[[Image:CDashMyCDash.jpg|thumb|MyCDash page]]


CDash support three levels of roles for users: Normal users, Site maintainers and Project administrator. Normal users are regular users with read and/or write access to the project’s code repository. Site maintainers are responsible for periodic submissions to CDash. Project administrators have reserved privileges to administer the project in CDash. The first two levels can be defined by the users themselves. Project administrators access has to be granted by another administrator of the project or CDash’s administrator.
Each user has its own personal section in CDash, where they can keep track of recent builds, submissions, claim sites and more.


In order to change a current role for a user:
== My Profile ==
# Select [Manage project roles] in the administration section.
# Then, if you have more than one project, select the appropriate project.
# In the “current users” section change the role for a user
# Click “update” to update the current role
# In order to remove completely a user from a project, click “remove”


If the cvs login is not correct it can also be changed from this page. Note that users can also change their CVS login manually from their profile.
User can change their profile: First/Last name, email, institution and password from the the "My Profile" menu at the top of the "My CDash" page.


In order to add a current role for a user:
== My Projects ==
# Select [Manage project roles] in the administration section.
# Then, if you have more than one project, select the appropriate project.
# In the “Add new user” section type the first letters of the first name, last name or email address of the user you want to add. Or type ‘%’ in order to show all the users registered in CDash.
# Select the appropriate user’s role
# Optionally put the user’s cvs login
# Click on “add user”


== Importing from CVS users file ==
If a user has subscribe to at least a project (see [[CDash:Documentation#Subscribe_to_project | Subscribe to project]]), the list of subscribed projects is shown under "My Projects". There several links related to the subscriptions.
In some cases, it is useful to batch import a list of current users for a project. In order to import:


# Click on [manage project role] in the administration section
=== Edit Subscription ===
# Select the appropriate project (if more than one)
# Click “Browse” to select a CVS users file.
The current file should be formatted as follow:
  cvsuser:email:FirstName LastName


# Click “import”
Users can edit their subscription for a given project.
# Make sure the reported names and email address are correct and deselect the ones that shouldn’t imported and click on “Register and send email”. This will automatically register the users, set a random password and send a registration request to the appropriate email addresses.


= Build Groups =
# From MyCdash page, under My Projects, click on [Edit subscription]
(also called tracks in Dart2)
# Change the role if necessary. '''Downgrading from project administrator to Site maintainer or normal user cannot be reverted unless by another project administrator'''.
# Change or add CVS/SVN login
# Change the email preferences if necessary
# Click on "Update Subscription" to validate the modifications


Builds can be organized by groups.  
Users can unsubscribe from a project by clicking on the "Unsubscribe" button. Users will be able to subscribe later on.
In CDash, three groups are defined automatically and cannot be removed: Nightly, Continuous and Experimental. These groups are the same imposed by CTest.


Each group has a description associated with it that is displayed when clicking on the name of the group on the main dashboard. In order to change the description see the section “update group description” below.
=== Claim sites ===
See [[CDash:Documentation#Claim a site | Claim a site]] section.


== Adding a new group ==
=== Edit project ===
See [[CDash:Administration#Editing_a_project | Editing a project]] section.


# Click on [manage project groups] in the administration section
=== Manage project groups ===
# Select the appropriate project (if more than one)
See [[CDash:Administration#Build_Groups | Build groups]] section.
# Under the “create new group” section enter the name of the new group
# Click on “create group”


Newly create group appears at the bottom of the current dashboard. In order to reorder the groups, see the next section.
=== Manage project roles ===
See [[CDash:Administration#Project_Roles | Project roles]] section.


== Ordering groups ==
== My Sites ==


# Click on [manage project groups] in the administration section
[[Image:CDashMySites.jpg|thumb|My sites]]
# Select the appropriate project (if more than one)
# Under the “Current Groups” section, click on the [up] or [down] links.


Note: the order displayed in this page is exactly the same at the order on the dashboard.
As a site maintainer, it is useful to know if submissions have been missing for quite sometime, or if the current builds on a specific sites are not reporting errors due to misconfiguration.


== Update group description ==
* The first column show the list of currently claimed sites.
* The first row shows the current projects that are submitting builds for the given site
* For each pair (site/project) a summary of the current status of the dashboard is displayed
{| border="1"
|- bgcolor="#abcdef"
! Build group !! Updates !! Errors !! Warnings !! Tests !! Last submission
|-
| N: Nightly || 1 || 0 || 0 || 0 || Today
|-
| C: Continuous || 1 || 0 || 0 || 0 || Yesterday
|-
| E: Experimental || 1 || 0 || 0 || 0 || 6 days ago
|}


# Click on [manage project groups] in the administration section
''Note: only Nightly, Continuous and Experimental groups are current shown'.'
# Select the appropriate project (if more than one)
# Under the “Current Groups” section, update or add a description in the field next to the [up][down] links.
# Click “Update Description” in order to commit your changes.


== Move builds to another group ==
Clicking on the name of a site gives access to the description of the site.


By default builds are going to the group associated with the build type defined by CTest, i.e. a nightly build will go in the nightly section.
== Public projects ==
There are two ways to move a group into the proper group and define a rule, such that if a same build is submitted it goes to the appropriate group: Global Move and Single Move.


CDash matches the builds by Name, Site, and Build Type. For instance if the build named “Linux-gcc-4.3” from the site “midworld.kitware” is a nightly build it will be move to the nightly section unless a rule on “Linux-gcc-4.3”-“midworld.kitware”-Nightly is defined.
List of all the public projects stored in CDash. By definition, users registered in CDash can subscribe to any public projects.


=== Global Move ===
= Subscribe to project =


Global move allows to move builds as a batch.
[[Image:CDashSubscribeProject.jpg|thumb|Subscribing to public projects]]


# Click on [manage project groups] in the administration section
If the project is public then users can subscribe to the project themselves, otherwise see section [[CDash:Administration#Project_Roles | Project Roles]].
# Select the appropriate project (if more than one)
# Under “Global Move” you will see a list of all the builds submitted in the past 7 days. (without duplicates). Note that expected builds are also shown, even if they have not been submitting in the past 7 days.
# You can narrow your search by selecting a specific group (default is All).
# Select the builds to move. Hold “shift” in order to select multiple builds.
# Select the target group. This is mandatory.
# Optionally check the “expected” box if you expect the builds to be submitted on a daily basis. For more information on expected builds see the “Expected builds” section below.
# Click “Selected Builds to Group” to move the groups.


=== Single Move ===
# Log into CDash (see [[CDash:Documentation#Login_to_CDash | Login to CDash]])
# Under "Public projects" section click on [Subscribe to this project]
# Select your role for the project: '''Normal user''' or '''Site maintainer''' if you are responsible for a machine sending periodic builds to the dashboard/
# Add your CVS/SVN login for the project if you have any
# Select your email preference
# Click on subscribe


From the main dashboard page, if logged as administrator of the project, a small folder icon will be present next to each build. Clicking on the icon will show some option for this build. In particular users are able to mark a build as expected (see section below) and move a build to a particular group or delete a bogus build.
= Claim a site =
[[Image:CDashClaimSite.jpg|thumb|Claiming a site]]


== Expected builds ==
CDash allows '''Site maintainers''' (see [[CDash:Documentation#Subscribe_to_project | Subscribe to Project]]) to claim a site they are managing. This allows users to have personal reports regarding a specific machine and get informed if the machine has not been submitting from quite some time or has unexpected high number of failures (probably due to misconfiguration).


Project administrators can mark certain builds as expected. That means builds are expected to submit daily.
In order to claim a site:
This allows to quickly check if a build has not been submitting on today’s dashboard or even by clicking on the info icon on the main dashboard to quickly assess for how long the build as been missing.


Note: CDash is going to send email when a build is missing in the near future.
# Login to CDash to acces "My CDash"
# Under "My Projects" select [Claim sites]. If the project is not listed, refers to [[CDash:Documentation#Subscribe_to_project | Subscribe to Project]] for more information.
# The Claim sites page show the list of current sites for the project. Check the site(s) of interest.
# Click on "Update claimed sites"


= Logging management  =
In order to unclaim a site:


CDash supports internal logging mechanism using the error_log() function from PHP. Most of the critical SQL queries (almost all the queries) are logged if an error occurs. By default CDash log file is located in the $CDASH_BACKUP_DIRECTORY as cdash.log. Note that CDash doesn’t perform any log rotation. The location of the log file can be modified by changing the  $CDASH_LOG_FILE variable in the config.php configuration file.
# Login to CDash to acces "My CDash"
# Under "My Projects" select [Claim sites].
# Uncheck the site(s) no longer of interest.
# Click on "Update claimed sites"


The log file can be access directly from CDash if the log file is in the standard location.
See the section [[CDash:Documentation#Describe_a_site | Describe a site]] for information about the given sites.


# Log into CDash as administrator
= Describe a site =
# Click on [CDash logs] in the administration section
[[Image:CDashSiteSpecifications.jpg|thumb|Describing a site]]
# Click on cdash.log to see the log file.
[[Image:CDashSite.jpg|thumb|Site information]]


For developers, these two functions should be used to add logging mechanism into CDash.
CDash records each site specifications so that users can quickly assess the type of machine, OS, memory, etc...


  function add_log($text,$function)
'''If using the CTest 2.6 (or higher) the site specifications are automatically sent as part of the submission.'''
  function add_last_sql_error($functionname)


= Backup mechanism =
Here's the list of specification currently present:


CDash backups all the XML submissions for a certain period of time. The default timeframe is 48hours. The timeframe can be changed in the config.php file using the  
*'''Name:''' '''This value shouldn't be changed, unless the site name sent by the build was wrong. Make sure the name of the build matches CTest buildname otherwise a new site will be created.'''
$CDASH_BACKUP_TIMEFRAME variable.
*'''64 bits:''' Is the machine running a 64bits OS
*'''Processor vendor:''' vendor name of the processor (i.e. GenuineIntel)
*'''Processor vendor ID:''' vendor identification of the processor (i.e. Intel Corporation)
*'''Processor family ID:''' family identification number of the processor (i.e. 15)
*'''Processor model ID:''' model identification number of the processor, usually depending on the processor family (i.e. 3)
*'''Processor cache size:''' size of the processor cache in MB.
*'''CPU Speed (MHz):''' speed of the CPU in MHz.
*'''Number of logical CPUs:''' Number of logical CPU per physical CPU.
*'''Number of physical CPUs:''' Number of physical CPU.
*'''Logical Processor per Physical:'''Number of logical processor per physical
*'''Total virtual memory (MB):''' Total virtual memory in MB.
*'''Total physical memory (MB):''' Total physical memory (RAM) in MB.
*'''Description:''' Description of the machine.
*'''IP address:''' Current IP address. This is filled in automatically by CDash
*'''Latitude:''' Geolocation from the IP address
*'''Longitude:''' Geolocation from the IP address


Note 1: If projects are private it is recommended to set the backup directory outside of the apache root directory to make sure that nobody can access the XML files.
'''CDash keeps an history''' of the machine description, in case of memory or processor upgrade. This is particularly useful to assess why a test is now running much faster than before. If the system has been upgraded, the checkbox "Force new description revision" should be checked. Note that CDash should automatically detect if a site has been upgraded.
Note 2: The backup directory is emptied only when a new submission arrives.


== Importing from backups ==
''Note: support for other specifications, such as graphic card information and more detailed information might be added in the near future.''


If necessary, CDash can automatically import builds from the Backup directory.
= Graphs =


# Log into CDash as administrator
CDash relies on the flot javascript library for graphing. See the section [[CDash:Design#Graphs | Graphs]] for information.
# Click on [Import from backups] in the administration section
CDash curently plots three types of graph as explained below.
# Click on “Import backups”.


= Import Dart1 files =
== Build time graph ==
[[Image:CDashBuildGraph.jpg|thumb|Build time graph]]


CDash supports importing from Dart1. CDash doesn’t support from Dart2 because Dart2 doesn’t have any ways to export the database as XML (as far as we know).
The time requires to build a project is recorded as part of CTest. Sometimes it might be useful to track the build time overtime. In order to do so:


# Log into CDash as administrator
# Go to the main dashboard for the project
# Click on [Import Dart1 Files] in the administration section
# Click on the build name you want to track
# Select the project to import to
# On the build summary page, click on [Show Build Time Graph], this performs an Ajax request to the database.
# Select the path to the Dart1 "Sites" directory for the project on the server. Note that the filesystem should be accessible from the webserver user.
# Select the appropriate date range
# Click on import


Note that the import mechanism might take a long time.
== Test time graphs ==


= Backward Compatibility Tools =
CDash plots graphs of the time it takes to run a specific test as well as its status (passed/failed) overtime.


Several tools are provided for backward compatibility and administrative purposes.
To look at these graphs:


== Assign builds to default group ==
# Go to the main dashboard for a project
If some builds are not assign to any groups for some unknown reasons (this should be really rate), you can assign the builds to their group based on their build type (Nightly, Experimental  or Continuous).
# Click on the number of test passed or failed
# From the list of tests, click on the status of the test.
# Click on [Show Test Time Graph] and/or [Show Failing/Passing Graph]


== Fix build groups based on build rules ==
If some group rules are defined but the builds are not going to the proper group (this should be really rate),  you can fix this by running this script.


== Delete builds with wrong dates ==
Some builds might be coming into CDash with the wrong date, because either the date in the XML file is not correct or the timezone is not recognized by CDash (mainly by PHP). These builds will not show up in any dashboard because the starting time is bogus. It is recommended to delete these builds.


== Compute test timing ==
= Adding notes to a build =
See the “recomputed test timing” section for more information.
[[Image:CDashAddNote.jpg|thumb|Adding a note to a build]]
In some cases, it is useful to tell other developers that you are currently looking at the errors for a builds and informed them. CDash implements a simple note mechanism.


== Upgrade CDash ==
* Login to CDash
See the upgrade CDash section for more information
* On the dashboard project page, click on the build name to which you want to add the note.
* Click on [Add a Note to this Build] link, located next to the current build matrix (see thumbnail)
* Enter a small message corresponding to the note
* Select the status of the note: '''Simple note''', '''Fix in progress''', '''Fixed'''.
* Click on "Add Note"


= Build summary =


= My CDash =
CDash summarizes the build errors and warnings on a single page. The errors are grouped by file and line number. It makes easier to fix source code.
In order to see the build summary:


Each user has its own personal section in CDash, where they can keep track of recent builds, submissions, claim sites and more.
# Go to the main dashboard for the project
# In the main menu select Dashboard->Builds


== My Projects ==
= Filters =
Some CDash pages allow you to use '''filters''' to narrow down the results to what's important to you.


== Filters on index.php ==
{| border="1"
|- bgcolor="#abcdef"
! Name !! Description !! Example Value
|-
| Build Duration || How long the build took to complete || 14m 58s
|-
| Build Errors || How many errors occurred during the build step || 1
|-
| Build Warnings || How many warnings occurred during the build step || 5
|-
| Build Name || The name of the build || Linux-cxx
|-
| Build Stamp || The CTest-generated TAG for this build || 20090223-0100-Nightly
|-
| Build Start Time || When this build started (including any update/configure/etc) || 2009-02-23 01:00:00 UTC
|-
| Build Type || What type of build this is || Experimental
|-
| Configure Duration || How long this build took to configure || 2m 45s
|-
| Configure Errors || How many errors occurred during the configure step || 2
|-
| Configure Warnings || How many warnings occurred during the configure step || 3
|-
| Expected || Whether or not this build has been marked as expected || true
|-
| Group || The name of this build's group || Nightly
|-
| Has Coverage || Whether or not this build performed coverage || true
|-
| Has CTest Notes || Whether or not any notes were uploaded with this build || false
|-
| Has Dynamic Analysis || Whether or not this build performed dynamic analysis || true
|-
| Has User Notes || Whether or not a CDash user left a note on this build || true
|-
| Label || Search for builds with a given label || vtkCommonCore
|-
| Revision || The version of the source code that was built (git SHA1, SVN version, etc) || e19dbfa
|-
| Site || Search for builds from a given site || kamino
|-
| Submission Client || What client was used to submit this build || ctest-3.0
|-
| SubProjects || Whitelist or blacklist SubProject results || my-subproject-name
|-
| Tests Duration || How long this build took to test || 5m 23s
|-
| Tests Failed || How many tests failed || 4
|-
| Tests Not Run || How many tests couldn't be run || 10
|-
| Tests Passed || How many tests passed || 20
|-
| Tests Timing Failed || [[CDash:Design#Test_Timing|How many tests failed the time status check]] || 8
|-
| Update Duration || How long this build took to update || 45s
|-
| Updated Files || How many source files were updated before this build || 5
|}


== Filters on queryTests.php ==
{| border="1"
|- bgcolor="#abcdef"
! Name !! Description !! Example Value
|-
| Build Name || The name of the build that performed this test || Linux-c++
|-
| Build Start Time || When this build started (including any update/configure/etc) || 2009-02-23 01:00:00 UTC
|-
| Details || Information about how this test exited || OTHER_FAULT
|-
| Label || Search for tests with a given label || vtkCommonCore
|-
| Site || Search for tests run on a given site || kamino
|-
| Status || Search for tests with a given status || passed
|-
| Test Name || The name of the test || kwsys.testHashSTL
|-
| Time || How long this test took to run (in seconds) || 12.73
|}


== Filters on testOverview.php ==
{| border="1"
|- bgcolor="#abcdef"
! Name !! Description !! Example Value
|-
| Build Name || The name of the build(s) whose tests you're interested in || Linux-c++
|}


== My Sites ==
== Filters on viewCoverage.php ==
{| border="1"
|- bgcolor="#abcdef"
! Name !! Description !! Example Value
|-
| Covered Lines || The number of covered lines in this file || 50
|-
| Filename || The name of this file || foo.cxx
|-| Labels || Search for source files with a given label || vtkCommonCore
|-
| Priority || The priority of this source file || 2
|-
| Total Lines || The number of coverable lines in this file || 100
|-
| Uncovered Lines || The number of uncovered lines in this file || 25
|}


As a site maintainer, it is useful to know if submissions have been missing for quite sometime.
== Filters on viewTest.php ==
{| border="1"
|- bgcolor="#abcdef"
! Name !! Description !! Example Value
|-
| Details || Information about how this test exited || OTHER_FAULT
|-
| Label || Search for tests with a given label || vtkCommonCore
|-
| Status || Search for tests with a given status || passed
|-
| Test Name || The name of the test || kwsys.testHashSTL
|-
| Time || How long this test took to run (in seconds) || 12.73
|-
| Time Status || [[CDash:Design#Test_Timing|Whether or not this build failed the time status check]] || 1
|}

Latest revision as of 17:50, 27 October 2016

< CDash Main Page

This page is the official CDash documentation.

CDash definitions

  • Build: Single submission to the dashboard for a specific project, environment and build type.
  • Build type: reference by CTest as Nightly, Experimental or Continuous. (see CMake documentation for more information).
  • Site: Computer contributing builds to the dashboard. A site might belong to several projects and submit different build.
  • Coverage: Number of lines of code actually tested. CDash/CTest currently supports gcov and bulleyes tools.
  • Dynamic Analysis: reports the memory allocation/deallocations analysis for test, examples and executable for the project. Valgrind is currently supported by CTest/CDash.
  • Nightly start time: starting time of the 24h window of the daily dashboard.
  • Ajax: Asynchronous JavaScript and XML, is a group of web development techniques used for creating interactive web applications.
  • PHP: Hypertext Preprocessor, is a computer scripting language, designed for producing dynamic web pages.
  • SQL: Structured Query Language, is a database computer language designed for the retrieval and management of data in relational database management systems.

Login to CDash

CDash uses secure login for users and adminsitrators. CDash does not store any plain text passwords in the database, therefore passwords cannot be recovered, even by system administrators.

Note: LDAP is not currently supported.

  • To login, click on the 'Login' link at the top of the main CDash page or directly go to the user.php page http://myserver/CDash/user.php.
  • If you don't have a login, click on the 'register' link on the top menu. In order to register, CDash requires a valid email address, your first name, last name and password. The institution is optional. CDash implements some security features to avoid robots to register.
  • Once registered you are ready to type your email and password provided in the login screen.

Available Dashboards

CDash index table

The first page display on CDash is a list of current public projects (private projects are not shown on this page). Several information are shown on this page:

  • The name of the project
  • The total number of submissions for each project
  • The date of the first build for each project
  • The date of the last submission (not build time)
  • To total size of the database

Clicking on the name of the project redirects the user to the main dashboard for this project.

Project Dashboard

Project's dashboard page
Help link from the main dashboard page

Each project dashboard have a similar layout:

  • Top row of the page:
    • Link to "My CDash/Login"
    • Current date of the server hosting CDash
    • Link to RSS feed to the project (if not private)
  • Banner:
    • Custom project logo (see Updating project's logo to upload a new logo). Note: clicking on the logo redirects to the project's home page (if any defined)
    • Project's name
  • Main Menu
    • Dashboard: goes back to the list of projects hosted on CDash
      • Updates: Overview of the updates for this day and this project
      • Builds: Overview of the builds for this day and this project
      • Tests: Overview of the test failing for this day and this project
      • Map: Map of the sites for this project and this day
    • Calendar: display a calendar to check previous dashboards
    • Previous: Go to the previous day
    • Current: Go to the current day
    • Next: Go to the next day
    • Project: Project links
      • Home: Home URL
      • Doxygen: Documentation page for this project
      • CVS: Link to the CVS/SVN viewer for this project
      • Bugs: Link to the bug tracker for this project
  • Daily updates: Updates on the project's repository.
  • Help link: popup a window with help information
  • Build groups: explained in the next section.
  • Coverage: explained in the next section.
  • Dynamic Analysis: explained in the next section.

Build groups

Each build submitted by a site corresponds to a row in the main dashboard. By default the build falls in the group defined by the type of submission (i.e Nightly, Experimental or Continuous), however, custom builds can be defined. See Build Groups administration section for more information on how to create new groups and define rules for builds.

  • Mouse hover the top row of a build group, shows the quick link bar to quickly jump to a different group.
  • Expected builds are shown as a empty row in the dashboard if they have not been submitting.

Default group

Muli-columns sorting
Group description
Note for a build

In this section we explain the layout of default groups (i.e. Nightly, Experimental, Continuous,...). These groups are represented in the light blue sections.

Clicking on the gray header allow sorting the table by the clicked field. Multisorting is possible by holding the shift key., also the current sorting is saved by the user's web browser using cookies, allowing to keep the same sorting while browsing different days for the same project.

Clicking on the group name displays the build group description.

Each build (row) contains the following information:

  • Site: Name of the site submitting the build. Clicking on the name redirects to site description.
  • Build name: Name of the build. Clicking on the name redirects to the build summary page.
    • Build note: If submitted with CTest and configure with build notes, a small icon shows the build note. Clicking on the icon displays the CTest script used to submit the page.
    • CTest version: If submitted with CTest, a small icon displays the CTest version.
    • Build info: If a build is expected and has not been submitting, or if a build has errors or warnings or tests failing, then a build info icon is present. Clicking on the build info icon trigger an ajax request that displays more information about this build.
    • Notes: If someone submitted a note for this build, mouse hover the note icon will display the current notes.
  • Update: Shows the number of files updated (from the SVN/CVS repository) for this build. Clicking on the number shows the name and information of the updated files.
  • Cfg: Shows the configuration errors. If red, some errors occurred while configuring the project. Clicking on the number displays the current configuration status.
  • Build: Displays the principal build information.
    • Error: Number of build errors.
    • Warn: Number of build warnings.
    • Min: Time (in minutes) it took to build the project.
  • Test: Displays information about the test.
    • Not Run: Number of test not run (because not found for instance).
    • Fail: Number of test failed. Clicking on the number show the name of the failing tests. If time status is enabled for the project, it is reflected here.
    • Pass: Number of test passed. Clicking on the number show the name of the passing tests. If time status is enabled for the project, it is reflected here.
    • Min: Time it took to run all the tests.
  • Build time: Time when the build started expressed in the timezone of the webserver. This is different from the time when the build was submitted.

Coverage

Coverage section of the dashboard

CDash supports gcov and bullseye as coverage tools. The main dashboard displays the following information:

  • Site: name of the site
  • Build name: name of the build
  • Percentage: Overall coverage percentage (for all files in the project).
  • Passed: Number of lines of code that are tested.
  • Failed: Number of lines of code that are not tested.
  • Date: Build starting date.

Dynamic Analysis

Dynamic Analysys section of the dashboard

CDash supports valgrind as the main dynamic analysis tool. The main dashboard displays the following information:

  • Site: Name of the site
  • Build name: Name of the build
  • Checker: Name of the checker.
  • Defect count: Number of memory defects.
  • Date: Build starting date.

Daily updates

CDash stores the daily updates in the database for a quick query. The number of files changed for that day as well as the number of authors who changed the files are displayed. Clicking on the number of files changed display the name of the files and more information.

See the Daily Updates Design section for more information.

Map

Map of the sites contributing to a project

CDash stores the geolocation from the IP address for the sites and displays them on a map via Google map API. This allows to quickly browse where the builds are coming from. Clicking on a pin on the map, shows the name of the site.

See Map and Geolocation design page for more information.

My CDash

MyCDash page

Each user has its own personal section in CDash, where they can keep track of recent builds, submissions, claim sites and more.

My Profile

User can change their profile: First/Last name, email, institution and password from the the "My Profile" menu at the top of the "My CDash" page.

My Projects

If a user has subscribe to at least a project (see Subscribe to project), the list of subscribed projects is shown under "My Projects". There several links related to the subscriptions.

Edit Subscription

Users can edit their subscription for a given project.

  1. From MyCdash page, under My Projects, click on [Edit subscription]
  2. Change the role if necessary. Downgrading from project administrator to Site maintainer or normal user cannot be reverted unless by another project administrator.
  3. Change or add CVS/SVN login
  4. Change the email preferences if necessary
  5. Click on "Update Subscription" to validate the modifications

Users can unsubscribe from a project by clicking on the "Unsubscribe" button. Users will be able to subscribe later on.

Claim sites

See Claim a site section.

Edit project

See Editing a project section.

Manage project groups

See Build groups section.

Manage project roles

See Project roles section.

My Sites

My sites

As a site maintainer, it is useful to know if submissions have been missing for quite sometime, or if the current builds on a specific sites are not reporting errors due to misconfiguration.

  • The first column show the list of currently claimed sites.
  • The first row shows the current projects that are submitting builds for the given site
  • For each pair (site/project) a summary of the current status of the dashboard is displayed
Build group Updates Errors Warnings Tests Last submission
N: Nightly 1 0 0 0 Today
C: Continuous 1 0 0 0 Yesterday
E: Experimental 1 0 0 0 6 days ago

Note: only Nightly, Continuous and Experimental groups are current shown'.'

Clicking on the name of a site gives access to the description of the site.

Public projects

List of all the public projects stored in CDash. By definition, users registered in CDash can subscribe to any public projects.

Subscribe to project

Subscribing to public projects

If the project is public then users can subscribe to the project themselves, otherwise see section Project Roles.

  1. Log into CDash (see Login to CDash)
  2. Under "Public projects" section click on [Subscribe to this project]
  3. Select your role for the project: Normal user or Site maintainer if you are responsible for a machine sending periodic builds to the dashboard/
  4. Add your CVS/SVN login for the project if you have any
  5. Select your email preference
  6. Click on subscribe

Claim a site

Claiming a site

CDash allows Site maintainers (see Subscribe to Project) to claim a site they are managing. This allows users to have personal reports regarding a specific machine and get informed if the machine has not been submitting from quite some time or has unexpected high number of failures (probably due to misconfiguration).

In order to claim a site:

  1. Login to CDash to acces "My CDash"
  2. Under "My Projects" select [Claim sites]. If the project is not listed, refers to Subscribe to Project for more information.
  3. The Claim sites page show the list of current sites for the project. Check the site(s) of interest.
  4. Click on "Update claimed sites"

In order to unclaim a site:

  1. Login to CDash to acces "My CDash"
  2. Under "My Projects" select [Claim sites].
  3. Uncheck the site(s) no longer of interest.
  4. Click on "Update claimed sites"

See the section Describe a site for information about the given sites.

Describe a site

Describing a site
Site information

CDash records each site specifications so that users can quickly assess the type of machine, OS, memory, etc...

If using the CTest 2.6 (or higher) the site specifications are automatically sent as part of the submission.

Here's the list of specification currently present:

  • Name: This value shouldn't be changed, unless the site name sent by the build was wrong. Make sure the name of the build matches CTest buildname otherwise a new site will be created.
  • 64 bits: Is the machine running a 64bits OS
  • Processor vendor: vendor name of the processor (i.e. GenuineIntel)
  • Processor vendor ID: vendor identification of the processor (i.e. Intel Corporation)
  • Processor family ID: family identification number of the processor (i.e. 15)
  • Processor model ID: model identification number of the processor, usually depending on the processor family (i.e. 3)
  • Processor cache size: size of the processor cache in MB.
  • CPU Speed (MHz): speed of the CPU in MHz.
  • Number of logical CPUs: Number of logical CPU per physical CPU.
  • Number of physical CPUs: Number of physical CPU.
  • Logical Processor per Physical:Number of logical processor per physical
  • Total virtual memory (MB): Total virtual memory in MB.
  • Total physical memory (MB): Total physical memory (RAM) in MB.
  • Description: Description of the machine.
  • IP address: Current IP address. This is filled in automatically by CDash
  • Latitude: Geolocation from the IP address
  • Longitude: Geolocation from the IP address

CDash keeps an history of the machine description, in case of memory or processor upgrade. This is particularly useful to assess why a test is now running much faster than before. If the system has been upgraded, the checkbox "Force new description revision" should be checked. Note that CDash should automatically detect if a site has been upgraded.

Note: support for other specifications, such as graphic card information and more detailed information might be added in the near future.

Graphs

CDash relies on the flot javascript library for graphing. See the section Graphs for information. CDash curently plots three types of graph as explained below.

Build time graph

Build time graph

The time requires to build a project is recorded as part of CTest. Sometimes it might be useful to track the build time overtime. In order to do so:

  1. Go to the main dashboard for the project
  2. Click on the build name you want to track
  3. On the build summary page, click on [Show Build Time Graph], this performs an Ajax request to the database.

Test time graphs

CDash plots graphs of the time it takes to run a specific test as well as its status (passed/failed) overtime.

To look at these graphs:

  1. Go to the main dashboard for a project
  2. Click on the number of test passed or failed
  3. From the list of tests, click on the status of the test.
  4. Click on [Show Test Time Graph] and/or [Show Failing/Passing Graph]


Adding notes to a build

Adding a note to a build

In some cases, it is useful to tell other developers that you are currently looking at the errors for a builds and informed them. CDash implements a simple note mechanism.

  • Login to CDash
  • On the dashboard project page, click on the build name to which you want to add the note.
  • Click on [Add a Note to this Build] link, located next to the current build matrix (see thumbnail)
  • Enter a small message corresponding to the note
  • Select the status of the note: Simple note, Fix in progress, Fixed.
  • Click on "Add Note"

Build summary

CDash summarizes the build errors and warnings on a single page. The errors are grouped by file and line number. It makes easier to fix source code. In order to see the build summary:

  1. Go to the main dashboard for the project
  2. In the main menu select Dashboard->Builds

Filters

Some CDash pages allow you to use filters to narrow down the results to what's important to you.

Filters on index.php

Name Description Example Value
Build Duration How long the build took to complete 14m 58s
Build Errors How many errors occurred during the build step 1
Build Warnings How many warnings occurred during the build step 5
Build Name The name of the build Linux-cxx
Build Stamp The CTest-generated TAG for this build 20090223-0100-Nightly
Build Start Time When this build started (including any update/configure/etc) 2009-02-23 01:00:00 UTC
Build Type What type of build this is Experimental
Configure Duration How long this build took to configure 2m 45s
Configure Errors How many errors occurred during the configure step 2
Configure Warnings How many warnings occurred during the configure step 3
Expected Whether or not this build has been marked as expected true
Group The name of this build's group Nightly
Has Coverage Whether or not this build performed coverage true
Has CTest Notes Whether or not any notes were uploaded with this build false
Has Dynamic Analysis Whether or not this build performed dynamic analysis true
Has User Notes Whether or not a CDash user left a note on this build true
Label Search for builds with a given label vtkCommonCore
Revision The version of the source code that was built (git SHA1, SVN version, etc) e19dbfa
Site Search for builds from a given site kamino
Submission Client What client was used to submit this build ctest-3.0
SubProjects Whitelist or blacklist SubProject results my-subproject-name
Tests Duration How long this build took to test 5m 23s
Tests Failed How many tests failed 4
Tests Not Run How many tests couldn't be run 10
Tests Passed How many tests passed 20
Tests Timing Failed How many tests failed the time status check 8
Update Duration How long this build took to update 45s
Updated Files How many source files were updated before this build 5

Filters on queryTests.php

Name Description Example Value
Build Name The name of the build that performed this test Linux-c++
Build Start Time When this build started (including any update/configure/etc) 2009-02-23 01:00:00 UTC
Details Information about how this test exited OTHER_FAULT
Label Search for tests with a given label vtkCommonCore
Site Search for tests run on a given site kamino
Status Search for tests with a given status passed
Test Name The name of the test kwsys.testHashSTL
Time How long this test took to run (in seconds) 12.73

Filters on testOverview.php

Name Description Example Value
Build Name The name of the build(s) whose tests you're interested in Linux-c++

Filters on viewCoverage.php

Name Description Example Value
Covered Lines The number of covered lines in this file 50
Filename The name of this file foo.cxx
Priority The priority of this source file 2
Total Lines The number of coverable lines in this file 100
Uncovered Lines The number of uncovered lines in this file 25

Filters on viewTest.php

Name Description Example Value
Details Information about how this test exited OTHER_FAULT
Label Search for tests with a given label vtkCommonCore
Status Search for tests with a given status passed
Test Name The name of the test kwsys.testHashSTL
Time How long this test took to run (in seconds) 12.73
Time Status Whether or not this build failed the time status check 1
Retrieved from "https://public.kitware.com/Wiki/index.php?title=CDash:Documentation&oldid=60962"