View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0011561CMakeCTestpublic2010-12-02 11:512011-05-02 14:45
Reportertyler 
Assigned ToZach Mullen 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformAllOSAllOS VersionAll
Product VersionCMake 2.8.3 
Target VersionCMake 2.8.4Fixed in VersionCMake 2.8.4 
Summary0011561: test property COST not working in cmake 2.8.3
DescriptionFull details in this thread:
http://www.mail-archive.com/cmake@cmake.org/msg32863.html [^]

The high points:

* Tyler: I use set_tests_properties (... COST -500) to force my code coverage
checks to run after all the unit tests have run. This has stopped
working.

* Zach: In this release we decided that the costs should only be taken into account in a parallel case (ctest -j N). Many users have implicit dependencies
based on the order of their add_test calls, so we didn't want to break
backward compatibility for those not using parallel ctest. The non-parallel way to specify a test to run last is simply to make it the last add_test call.

* Zach: The middle ground here would be to respect costs
in the non-parallel case when they are expressed explicitly, but not to
cost-order them automatically based on their previous run times.

* Tyler: Perhaps if CTest reserved a range for its own COST data (-10 <=
CTest-calculated COST <= 10?) then users could use costs < -10 or > 10
to insure ordering of certain tests?
Steps To Reproduce[tyle...@tpb006:~/cmake-test-properties-test]$ cat CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
project(p)
enable_testing()

add_test (i_should_run_second echo i should run second)
set_tests_properties (i_should_run_second PROPERTIES COST -1)
add_test (i_should_run_first echo i should run first)

[tyle...@tpb006:~/cmake-test-properties-test]$ mkdir b && cd b && cmake ..
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to:
/tpb006/tylermr/cmake-test-properties-test/b

[tyle...@tpb006:~/cmake-test-properties-test/b]$ ctest --version
ctest version 2.8.3
[tyle...@tpb006:~/cmake-test-properties-test/b]$ ctest
Test project /tpb006/tylermr/cmake-test-properties-test/b
    Start 1: i_should_run_second
1/2 Test #1: i_should_run_second .............. Passed 0.01 sec
    Start 2: i_should_run_first
2/2 Test 0000002: i_should_run_first ............... Passed 0.01 sec

100% tests passed, 0 tests failed out of 2

Total Test time (real) = 0.04 sec

[tyle...@tpb006:~/cmake-test-properties-test/b]$
/usr/local/cmake-2.8.1-Linux-i386/bin/ctest --version
ctest version 2.8.1
[tyle...@tpb006:~/cmake-test-properties-test/b]$
/usr/local/cmake-2.8.1-Linux-i386/bin/ctest
Test project /tpb006/tylermr/cmake-test-properties-test/b
    Start 2: i_should_run_first
1/2 Test 0000002: i_should_run_first ............... Passed 0.01 sec
    Start 1: i_should_run_second
2/2 Test #1: i_should_run_second .............. Passed 0.01 sec

100% tests passed, 0 tests failed out of 2

Total Test time (real) = 0.02 sec
TagsNo tags attached.
Attached Filestxt file icon COST-changes-patch.txt [^] (1,941 bytes) 2010-12-09 12:03 [Show Content]

 Relationships
related to 0011877closedDavid Cole ctest runs tests in different order than previous versions of ctest 

  Notes
(0023849)
tyler (reporter)
2010-12-09 12:03

I have created a patch that attempts to resolve this issue. The patch is attached. Further details at http://www.mail-archive.com/cmake@cmake.org/msg33119.html. [^]
(0024362)
Zach Mullen (developer)
2011-01-03 11:39

Applied the patch and created a test for the behavior:

http://cmake.org/gitweb?p=cmake.git;a=commit;h=6b0c7ded5777c5b4c703bbe59be5c6afcebb879d [^]

http://www.cdash.org/CDash/viewTest.php?onlydelta&buildid=816300 [^]
http://www.cdash.org/CDash/viewTest.php?onlydelta&buildid=816293 [^]
(0026302)
David Cole (manager)
2011-05-02 14:45

Closing resolved issues that have not been updated in more than 3 months.

 Issue History
Date Modified Username Field Change
2010-12-02 11:51 tyler New Issue
2010-12-09 12:03 tyler File Added: COST-changes-patch.txt
2010-12-09 12:03 tyler Note Added: 0023849
2010-12-10 13:25 David Cole Assigned To => Zach Mullen
2010-12-10 13:25 David Cole Status new => assigned
2010-12-10 13:26 David Cole Target Version => CMake 2.8.4
2011-01-03 11:39 Zach Mullen Note Added: 0024362
2011-01-03 11:39 Zach Mullen Status assigned => resolved
2011-01-03 11:39 Zach Mullen Fixed in Version => CMake 2.8.4
2011-01-03 11:39 Zach Mullen Resolution open => fixed
2011-02-18 12:02 David Cole Relationship added related to 0011877
2011-05-02 14:45 David Cole Note Added: 0026302
2011-05-02 14:45 David Cole Status resolved => closed


Copyright © 2000 - 2018 MantisBT Team