KWWidgets/GUI Testing/Squish/SquishForSlicer3

Goals

The goal here is to add Squish tests to Slicer3, the same was some were added to KWWidgets. If you have not checked Squish with KWWidgets or GUI Testing with Squish, please do so before you adding Squish tests to Slicer3.

Warning: Squish *must* be built with the same Tcl/Tk libraries that Slicer3 is using. Squish won't launch Slicer3 properly otherwise.

The code presented in the next few sections has not been commited to Slicer3 yet, but can be used as a starting point.

Files

Modify the following files in the Slicer3 source directory:

• Slicer3/CMake/SlicerMacros.cmake: define the Slicer3_ADD_Squish_TEST macro

# Slicer3_ADD_Squish_TEST

macro(slicer3_add_squish_test
    test_name
    squish_AUT_name
    squish_test_case_path
    slicerScriptFile
    )

  # Try to find the full path to the test executable

  include("${KWWidgets_CMAKE_DIR}/KWWidgetsTestingMacros.cmake")
  kwwidgets_get_full_path_to_executable(${squish_AUT_name} exe_path)
 
  KWWidgets_ADD_Squish_TEST(
    "${test_name}"
    "${exe_path}"
    "${squish_test_case_path}"
    "${slicerENV}"
    "${slicerPathsScript}"
    "--test-mode --script ${slicerScriptFile}"
    )

  SET_TESTS_PROPERTIES(${test_name}
    PROPERTIES FAIL_REGULAR_EXPRESSION "FAILED;ERROR;FATAL")

endmacro(slicer3_add_squish_test)

Usage

Create Squish test suites

Create a new Squish test suite under Slicer3/Applications/GUI/Testing/Squish with the Squish IDE to instrument the bin/Slicer3-real executable. Since Squish is not able to record the file name of an Open File Dialog at the moment (an issue being discussed with FrogLogic), we need to write a Tcl script that will load a scene and a few volume(s) for us. Here is a simple example of such a script, LoadSimpleScene.tcl:

update
$::slicer3::MRMLScene SetURL $::Slicer3_HOME/share/MRML/Testing/volScene.mrml
$::slicer3::MRMLScene Connect
update

Now copy this LoadSimpleScene.tcl file to Slicer3/Applications/GUI/Testing, go to "Test Suite -> Settings..." in the Squish IDE and modify the arguments of the AUT as follow: --test-mode --script ../../Slicer3/Applications/GUI/Testing/LoadSimpleScene.tcl. This will instruct Slicer3 to automatically run the script and open the corresponding scene when launched from Squish. Note: the --test-mode option is used to NoSplash=true and RegistryLevel=0.

Add Squish tests to CTest lists

Add the following to the CMakeLists.txt file under Slicer3/Applications/GUI/Testing, and here the LoadSimpleScene.tcl will then be passed to Squish tests as a AUT-start-script. NOTE: This is just a very simple example of adding a Squish test for Slicer, and for a full comprehensive GUI testing, there should be a lot of AUT-start-scripts (similar to the LoadSimpleScene.tcl) to cover all the data-loading scenarios with different datasets, and the following CMake script should be modified accordingly.

include("${KWWidgets_CMAKE_DIR}/FindSquish.cmake")

if(SQUISH_FOUND)

  include("${Slicer3_CMAKE_DIR}/Slicer3Macros.cmake")

  file(GLOB SquishTestSuites ${Slicer3_SOURCE_DIR}/Applications/GUI/Testing/Squish/* suite_*)

  foreach(SquishTestSuite ${SquishTestSuites})
    if(IS_DIRECTORY ${SquishTestSuite})

      string(REGEX MATCH ".+suite_(.*)" tmp_suite_name ${SquishTestSuite})
      set(squish_suite_name ${CMAKE_MATCH_1})  

      file(GLOB SquishTests ${SquishTestSuite}/* tst_*)
      foreach(SquishTest ${SquishTests})
        if(IS_DIRECTORY ${SquishTest})
        
          string(REGEX MATCH ".+(tst_(.*))" tmp_test_name ${SquishTest})
          set(squish_test_name "${CMAKE_MATCH_1}")  
          set(slicer_squish_test_name "Squish_${squish_suite_name}_${CMAKE_MATCH_2}")   

# message(STATUS "squish_test_name='${squish_test_name}'")
# message(STATUS "slicer_squish_test_name='${slicer_squish_test_name}'")
          
          slicer3_add_squish_test(
           ${slicer_squish_test_name} "${EXECUTABLE_OUTPUT_PATH}/Slicer3-real" 
           "${SquishTestSuite}/${squish_test_name}"
           "${Slicer3_SOURCE_DIR}/Applications/GUI/Testing/LoadSimpleScene.tcl")    

        endif(IS_DIRECTORY ${SquishTest})
      endforeach(SquishTest)
    endif(IS_DIRECTORY ${SquishTestSuite})
  endforeach(SquishTestSuite)

endif(SQUISH_FOUND)

Known Issues

Squish

• Squish issues with KWWidgets

Slicer3

• Screenshot to create baseline images and validate the Squish image-data-processing tests.