MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0009734CMakeModulespublic2009-10-19 13:282012-11-29 03:39
ReporterAlex Neundorf 
Assigned ToAlex Neundorf 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product VersionCMake-2-8 
Target VersionCMake 2.8.11Fixed in Version 
Summary0009734: FindSquish.cmake: SQUISH_ADD_TEST() macro does not do what the docs say
DescriptionThe SQUISH_ADD_TEST(), where the docs say it does
SQUISH_ADD_TEST(myTestName myApplication testSuiteName testCaseName)
doesn't do that, but instead does
SQUISH_ADD_TEST(testName testAUT testCase envVars testWraper)
.
I think this is not only a documentation bug, but instead the macro should be fixed to do what the docs say.
E.g. the current version cannot call a test executable with command line arguments (which could be done with the version which is documented).

Alex
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
Attached Files? FindSquish.cmake (8,231) 2011-07-14 15:59
https://public.kitware.com/Bug/file/3973/FindSquish.cmake
? SquishRunTestCase.sh (774) 2011-07-14 16:00
https://public.kitware.com/Bug/file/3974/SquishRunTestCase.sh
? SquishTestScript.cmake (3,340) 2011-07-14 16:00
https://public.kitware.com/Bug/file/3975/SquishTestScript.cmake
diff cmake-squish-windows.diff (2,871) 2011-08-17 10:11
https://public.kitware.com/Bug/file/4008/cmake-squish-windows.diff
Issue History
Date ModifiedUsernameFieldChange
2009-10-19 13:28Alex NeundorfNew Issue
2009-11-05 13:14Alex NeundorfFile Added: SquishTestScript.cmake
2009-11-05 13:15Alex NeundorfFile Added: FindSquish.cmake
2009-11-05 13:15Alex NeundorfFile Added: SquishRunTestCase.sh
2009-11-05 13:17Alex NeundorfNote Added: 0018351
2010-12-15 09:07David ColeStatusnew => assigned
2010-12-15 09:07David ColeAssigned To => Bill Hoffman
2011-07-12 09:43Volker KrauseNote Added: 0027027
2011-07-12 16:21Alex NeundorfNote Added: 0027031
2011-07-14 15:59Alex NeundorfFile Deleted: SquishTestScript.cmake
2011-07-14 15:59Alex NeundorfFile Deleted: FindSquish.cmake
2011-07-14 15:59Alex NeundorfFile Deleted: SquishRunTestCase.sh
2011-07-14 15:59Alex NeundorfFile Added: FindSquish.cmake
2011-07-14 16:00Alex NeundorfFile Added: SquishRunTestCase.sh
2011-07-14 16:00Alex NeundorfFile Added: SquishTestScript.cmake
2011-07-14 16:02Alex NeundorfNote Added: 0027036
2011-07-15 04:01Volker KrauseNote Added: 0027038
2011-07-15 14:26Alex NeundorfNote Added: 0027039
2011-07-16 03:08Volker KrauseNote Added: 0027043
2011-07-17 06:08Alex NeundorfNote Added: 0027051
2011-07-17 06:58Volker KrauseNote Added: 0027052
2011-08-17 10:11Volker KrauseFile Added: cmake-squish-windows.diff
2011-08-17 10:14Volker KrauseNote Added: 0027215
2012-11-04 09:13Alex NeundorfAssigned ToBill Hoffman => Alex Neundorf
2012-11-04 09:13Alex NeundorfTarget Version => CMake 2.8.11
2012-11-04 15:10Alex NeundorfNote Added: 0031417
2012-11-05 15:23Alex NeundorfNote Added: 0031463
2012-11-29 03:39Alex NeundorfNote Added: 0031758
2012-11-29 03:39Alex NeundorfStatusassigned => closed
2012-11-29 03:39Alex NeundorfResolutionopen => fixed

Notes
(0018351)
Alex Neundorf   
2009-11-05 13:17   
I attached the files I'm currently using, didn't get around to cleaning them up yet.
Also, since I don't have windows around, I did not touch the windows part, so this currently probably does not work with the files I attached here.

...but now the macro actually works and I can run any testcase from any testsuite in any directory, also containing spaces.

Alex
(0027027)
Volker Krause   
2011-07-12 09:43   
Alex' changes also made this work for me, I only needed to do one minor change to make it detect test failures correctly here (Squish 4.0.2): FAILED -> FAIL in the failure regular expression in FindSquish.cmake.
(0027031)
Alex Neundorf   
2011-07-12 16:21   
I'll attach a more recent verrsion the next days.
I did a few more minor changes in the meantime.
(0027036)
Alex Neundorf   
2011-07-14 16:02   
I removed the old attached files and uploaded our current ones.
The syntax of the squish_add_test() function has changed, it supports more features now:
squish_add_test(cmakeTestName AUT targetName SUITE suiteName
                           TEST squishTestName [SETTINGSGROUP group]
                           [PRE_COMMAND command] [POST_COMMAND command] )

Did you also try to use it under Windows ?
I only tested it under Linux, there it works fine.
Not sure how to integrate this in cmake, since it is incompatible to the existing FindSquish.
Maybe rename to FindSquish4 ?

Alex
(0027038)
Volker Krause   
2011-07-15 04:01   
The new version doesn't work without modifications for me. The --settingsgroup
option is deprecated in the Squish version I have here, so I had to remove that
from the .sh file to make things work again. I've tested on Linux only so far.

Volker
(0027039)
Alex Neundorf   
2011-07-15 14:26   
Oh, really ?
I'm using it with squish 4.0.2 and it is fine.
Which version do you have and what should be used instead a settings group ?

Alex
(0027043)
Volker Krause   
2011-07-16 03:08   
I'm using 4.0.2 as well. All I could find in the documentation about settings groups is that they are deprecated, unfortunately nothing about the replacement.

I tried making it work with the settings group argument, but no matter what I set there, Squish didn't want to start with that. Maybe something missing in my Squish test suite settings?

Anyway, I think it would be a good idea to make all non-mandatory arguments for Squish also optional on the CMake side (SETTINGSGROUP obviously, but also e.g. AUT and TESTCASE). This would allow the most flexibility, since it looks like Squish setups differ quite a bit.

Btw, why do we need the extra .bat/.sh file here, can't we directly call those commands from CMake using execute_process()? Would be less code and no platform differences anymore.

Volker
(0027051)
Alex Neundorf   
2011-07-17 06:08   
I agree about the settingsgroup, but how do you use it without AUT and without TESTCASE ?

Alex
(0027052)
Volker Krause   
2011-07-17 06:58   
If you don't specify the AUT it seems to take whatever is in the suite config or test scripts, probably not the best choice if you want code that works everywhere, but technically it seems to work just fine without that.

If you don't specify the testcase, it runs the entire suite. Again not optimal as the suite shows up as a single test in ctest output then, but technically this also works just fine. It would allow you to integrate a bunch of Squish tests with just a single CMake line though (instead of listing or iterating over all test cases), which can be useful if you need to get started quickly.

Volker
(0027215)
Volker Krause   
2011-08-17 10:14   
I've attached a patch to your scripts that adds Windows support. Note that the SquishRunTestCase.bat ignores the settings group argument right now, unlike the Unix version, adding that is straightforward though.
(0031417)
Alex Neundorf   
2012-11-04 15:10   
This is now in the MakeSquish4Work branch on cmake stage.
(0031463)
Alex Neundorf   
2012-11-05 15:23   
This is now merged into the next branch, so it should become part of cmake 2.8.11.
Please give it a try and let me know if there are issues with it.
(0031758)
Alex Neundorf   
2012-11-29 03:39   
This should be working as it is now in the master branch.
If there are issues with it, please reopen this one or enter a separate bug.