[CMake] confused by ctest
Bill O'Hara
billtohara at gmail.com
Tue Feb 3 15:13:27 EST 2009
Hello,
I wonder if anyone can point me in the direction of a hello world project
that successfully demonstrates the basic features of cmake+ctest+cdash? I'm
using cmake/ctest version 2.6-patch 2 and cdash 1.2.1. Or somewhere with a
working tutorial I can follow along? (I'm a bit confused by where to find
the up to date documentation I should work from).
I'm tasked with putting together a new build system for a commercial project
but when evaluating cmake I can't seem to get the basics working even on
trivial hello world projects. The comedy of errors I've managed so far is
included below - I'd be extremely grateful for any hints. I'm particularly
worried by the testing submitting results to kitware.com even though I
specifically said we're using cdash and pointed it at my local cdash server
(where some results are submitted despite other errors).
Thanks in advance for any hints! I have marked QUESTION below at various
points if anyone can spot my problems :(
Bill
#
# Start from a clean setup.
#
bill-box:/tmp> ls svn
README.txt conf dav db format hooks locks
bill-box:/tmp> ls build/
bill-box:/tmp> ls binary/
bill-box:/tmp> ls source/
#
# Checkout has a minimal test project checked out from /tmp/svn:
# - A shared library called impl build from include/header.h and impl/impl.c
# - A test called foo that invokes the function from impl and always
succeeds.
# - A test called bar that invokes the function from impl and always fails.
#
bill-box:/tmp> ls checkout/
CMakeLists.txt CTestConfig.cmake impl include tests
#
# CMakeLists.txt
#
bill-box:/tmp/checkout> cat CMakeLists.txt
PROJECT (HELLOWORLD)
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
INCLUDE_DIRECTORIES (${CMAKE_SOURCE_DIR}/include)
ADD_SUBDIRECTORY (${CMAKE_SOURCE_DIR}/impl)
INCLUDE (CTest)
ADD_SUBDIRECTORY (${CMAKE_SOURCE_DIR}/tests)
#
# CTestConfig.cmake
#
bill-box:/tmp/checkout> cat CTestConfig.cmake
SET (CTEST_DROP_SITE_CDASH TRUE)
SET (CTEST_PROJECT_NAME "helloworld")
SET (CTEST_NIGHTLY_START_TIME "00:00:00 EST")
SET (UPDATE_TYPE "true")
SET (CTEST_DROP_METHOD "http")
SET (CTEST_DROP_SITE "bill-box")
SET (CTEST_DROP_LOCATION "/submit.php?project=helloworld")
#
# And an experimental Nightly ctest script to try to upload
# test results to CDash after checking out and building
# the project source code. This script is not in /tmp/checkout.
#
bill-box:/tmp> cat experimental.cmake
SET (CTEST_DROP_SITE_CDASH TRUE)
SET (CTEST_PROJECT_NAME "helloworld")
SET (CTEST_NIGHTLY_START_TIME "00:00:00 EST")
SET (UPDATE_TYPE "true")
SET (CTEST_DROP_METHOD "http")
SET (CTEST_DROP_SITE "bill-box")
SET (CTEST_DROP_LOCATION "/submit.php?project=helloworld")
SET (CTEST_SOURCE_DIRECTORY "/tmp/source")
SET (CTEST_BINARY_DIRECTORY "/tmp/binary")
SET (CTEST_START_WITH_EMPTY_BINARY_DIRECTORY TRUE)
SET (CTEST_SVN_CHECKOUT "svn co file:///tmp/svn
\"${CTEST_SOURCE_DIRECTORY}\"")
SET (CTEST_CMAKE_GENERATOR "Unix Makefiles")
SET (CTEST_CUSTOM_PRE_TEST "${CTEST_SVN_CHECKOUT}")
CTEST_START(Nightly)
CTEST_UPDATE(SOURCE "${CTEST_SOURCE_DIRECTORY}")
CTEST_CONFIGURE(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_BUILD(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_TEST(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_SUBMIT()
#
# First, try to configure using cmake.
#
bill-box:/tmp> cd build
bill-box:/tmp/build> cmake /tmp/checkout
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/build
#
# Now, build and test.
#
bill-box:/tmp/build> make all test
Scanning dependencies of target impl
[ 33%] Building C object impl/CMakeFiles/impl.dir/impl.c.o
Linking C shared library libimpl.so
[ 33%] Built target impl
Scanning dependencies of target bar
[ 66%] Building C object tests/CMakeFiles/bar.dir/bar.c.o
Linking C executable bar
[ 66%] Built target bar
Scanning dependencies of target foo
[100%] Building C object tests/CMakeFiles/foo.dir/foo.c.o
Linking C executable foo
[100%] Built target foo
Running tests...
Start processing tests
Test project /tmp/build
1/ 2 Testing foo Passed
2/ 2 Testing bar ***Failed
50% tests passed, 1 tests failed out of 2
The following tests FAILED:
2 - bar (Failed)
Errors while running CTest
make: *** [test] Error 8
# QUESTION
#
# Is this Error 8 from make the expected result after some tests have
failed?
#
#
# Now try the Nightly test target.
#
bill-box:/tmp/build> make Nightly
Scanning dependencies of target Nightly
Site: bill-box
Build name: Linux-c++
Determine Nightly Start Time
Specified time: 00:00:00 EST
Create new tag: 20090203-0500 - Nightly
Start processing tests
#
# QUESTION
# Why does it think we are using CVS here?
#
Updating the repository
Updating the repository: /tmp/checkout
Use CVS repository type
Determine Nightly Start Time
Specified time: 00:00:00 EST
Gathering version information (each . represents one updated file):
Project is up-to-date
Update with command: "/usr/bin/cvs" -z3 update -d -A -P -D "2009-02-03
05:00:00 UTC" failed
Configure project
Each . represents 1024 bytes of output
. Size of output: 0K
Build project
Each symbol represents 1024 bytes of output.
'!' represents an error and '*' a warning.
. Size of output: 0K
0 Compiler errors
0 Compiler warnings
Test project /tmp/build
1/ 2 Testing foo Passed
2/ 2 Testing bar ***Failed
50% tests passed, 1 tests failed out of 2
The following tests FAILED:
2 - bar (Failed)
#
# QUESTION
# How do I tell it to not do Coverage?
#
Performing coverage
Cannot find any coverage files. Ignoring Coverage request.
Submit files (using http)
Using HTTP submit method
Drop site: http://bill-box/submit.php?project=helloworld
Uploaded: /tmp/build/Testing/20090203-0500/Build.xml
Uploaded: /tmp/build/Testing/20090203-0500/Configure.xml
Uploaded: /tmp/build/Testing/20090203-0500/Test.xml
Uploaded: /tmp/build/Testing/20090203-0500/Update.xml
Submission successful
Errors while running CTest
make[3]: *** [CMakeFiles/Nightly] Error 8
make[2]: *** [CMakeFiles/Nightly.dir/all] Error 2
make[1]: *** [CMakeFiles/Nightly.dir/rule] Error 2
make: *** [Nightly] Error 2
#
# QUESTION
# Why are we getting 3 different Errors from make?
#
#
#
# Let us try that again using ctest with verbosity turned on.
#
#
bill-box:/tmp/build> ctest -VV
#
# QUESTION
# Where is this DartConfiguration coming from? We're trying
# to use CDash.
#
UpdateCTestConfiguration from :/tmp/build/DartConfiguration.tcl
Parse Config file:/tmp/build/DartConfiguration.tcl
Start processing tests
UpdateCTestConfiguration from :/tmp/build/DartConfiguration.tcl
Parse Config file:/tmp/build/DartConfiguration.tcl
Test project /tmp/build
Constructing a list of tests
Done constructing a list of tests
Changing directory into /tmp/build/tests
1/ 2 Testing foo
Test command: /tmp/build/tests/foo foo.c
Test timeout computed to be: 1500
42
-- Process completed
Passed
2/ 2 Testing bar
Test command: /tmp/build/tests/bar bar.c
Test timeout computed to be: 1500
42
-- Process completed
***Failed
50% tests passed, 1 tests failed out of 2
The following tests FAILED:
2 - bar (Failed)
Errors while running CTest
#
#
# Try our experimental Nightly test script to check out and test the
project.
#
#
bill-box:/tmp> ctest -S experimental.cmake
#
# Drat; its complaining about the source control system and trying
# to build and update before running my svn checkout successfully.
#
# It also fails to find my tests.
#
Cannot find CVSCommand, SVNCommand, or UpdateCommand key in the
DartConfiguration.tcl
Error(s) when updating the project
Error(s) when building project
A /tmp/source/impl
A /tmp/source/impl/impl.c
A /tmp/source/impl/CMakeLists.txt
A /tmp/source/tests
A /tmp/source/tests/foo.c
A /tmp/source/tests/bar.c
A /tmp/source/tests/CMakeLists.txt
A /tmp/source/include
A /tmp/source/include/header.h
A /tmp/source/CTestConfig.cmake
A /tmp/source/experimental.cmake
A /tmp/source/CMakeLists.txt
Checked out revision 9.
No tests were found!!!
#
# QUESTION
# Why were my tests not found?
#
#
# Try that again with verbosity; the previous attempt seems to have
# helped by eventually checking out source code.
#
#
bill-box:/tmp> ctest -VV -S experimental.cmake
* Extra verbosity turned on
Reading Script: /tmp/experimental.cmake
SetCTestConfiguration:SourceDirectory:/tmp/source
SetCTestConfiguration:BuildDirectory:/tmp/binary
Run dashboard with model Nightly
Source directory: /tmp/source
Build directory: /tmp/binary
Reading ctest configuration file: /tmp/source/CTestConfig.cmake
SetCTestConfigurationFromCMakeVariable:NightlyStartTime:CTEST_NIGHTLY_START_TIMESetCTestConfiguration:NightlyStartTime:00:00:00
EST
Site:
Build name:
Determine Nightly Start Time
Specified time: 00:00:00 EST
Use Nightly tag: 20090203-0500
SetCTestConfiguration:SourceDirectory:/tmp/source
SetCTestConfiguration:BuildDirectory:/tmp/binary
SetCTestConfiguration:SourceDirectory:/tmp/source
Updating the repository
Updating the repository: /tmp/source
Cannot find CVSCommand, SVNCommand, or UpdateCommand key in the
DartConfiguration.tcl
SetCTestConfiguration:BuildDirectory:/tmp/binary
SetCTestConfiguration:SourceDirectory:/tmp/source
SetCTestConfiguration:ConfigureCommand:"/opt/cmake-2.6.2/bin/cmake" "-GUnix
Makefiles" "/tmp/source"
SetCTestConfiguration:BuildDirectory:/tmp/binary
SetCTestConfiguration:SourceDirectory:/tmp/source
Configure project
Configure with command: "/opt/cmake-2.6.2/bin/cmake" "-GUnix Makefiles"
"/tmp/source"
Run command: "/opt/cmake-2.6.2/bin/cmake" "-GUnix Makefiles" "/tmp/source"
Each . represents 1024 bytes of output
.-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/binary
Size of output: 0K
Command exited with the value: 0
SetMakeCommand:/usr/bin/gmake -i
SetCTestConfiguration:MakeCommand:/usr/bin/gmake -i
SetCTestConfiguration:BuildDirectory:/tmp/binary
SetCTestConfiguration:SourceDirectory:/tmp/source
Build project
MakeCommand:/usr/bin/gmake -i
Run command: "/usr/bin/gmake" "-i"
Each symbol represents 1024 bytes of output.
'!' represents an error and '*' a warning.
.[ 33%] Built target impl
[ 66%] Built target bar
[100%] Built target foo
Size of output: 0K
Command exited with the value: 0
0 Compiler errors
0 Compiler warnings
SetCTestConfiguration:BuildDirectory:/tmp/binary
SetCTestConfiguration:SourceDirectory:/tmp/source
Test project /tmp/binary
Run command: svn co file:///tmp/svn "/tmp/source"
Checked out revision 9.
Constructing a list of tests
Changing directory into /tmp/binary/tests
1/ 2 Testing foo
Test command: /tmp/binary/tests/foo foo.c
Test timeout computed to be: 600
42
-- Process completed
Passed
2/ 2 Testing bar
Test command: /tmp/binary/tests/bar bar.c
Test timeout computed to be: 600
42
-- Process completed
***Failed
50% tests passed, 1 tests failed out of 2
The following tests FAILED:
2 - bar (Failed)
* Use default trigger site:
http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi
SetCTestConfiguration:DropMethod:http
SetCTestConfiguration:DropSite:bill-box
SetCTestConfiguration:DropLocation:/submit.php?project=helloworld
SetCTestConfiguration:TriggerSite:
http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi
SetCTestConfiguration:BuildDirectory:/tmp/binary
SetCTestConfiguration:SourceDirectory:/tmp/source
Submit files (using http)
Using HTTP submit method
Drop site: http://bill-box/submit.php?project=helloworld
Upload file: /tmp/binary/Testing/20090203-0500/Build.xml to
http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Build.xmlSize:
909
Uploaded: /tmp/binary/Testing/20090203-0500/Build.xml
Upload file: /tmp/binary/Testing/20090203-0500/Configure.xml to
http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Configure.xmlSize:
1066
Uploaded: /tmp/binary/Testing/20090203-0500/Configure.xml
Upload file: /tmp/binary/Testing/20090203-0500/Test.xml to
http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Test.xmlSize:
2272
Uploaded: /tmp/binary/Testing/20090203-0500/Test.xml
#
# QUESTION
# What the...?
#
# Why is it submitting results to kitware.com?!
#
#
#
Using HTTP trigger method
Trigger site:
http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi
Trigger url:
http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Build.xml
Trigger url:
http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Configure.xml
Trigger url:
http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Test.xml
Dart server triggered...
Submission successful
bill-box:/tmp>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20090203/c5b4f8a7/attachment-0001.htm>
More information about the CMake
mailing list