[IGSTK-Users] State Machine Validation

[Kevin Gary]

Charles,

Thank you for your questions, I have tried to answer them below. I'd also be 
happy to work through any issues with your XML files if you post them.

Please repost if you need additional assistance.

Kevin


charles.garraud at irisa.fr wrote:
> Hi,
> 
> We would like to introduce state machine validation in our testing 
> process, but I think I miss something essential regarding the aim of the 
> validation
> tool included in the contribution package.
> 
> What we were thinking to do is to automatically:
> 
> - generate the .scxml file from the state machine
> - launch run-coverage on the scxml file to generate one .xml for the 
> node coverage and one .xml for the edge coverage
> - launch run-simulator with a config file like the following:
> <vtconfig>
> <drool-files>
> <file>rules/igstk_1.drl</file>
> </drool-files>
> <state-machines>
> <state-machine id="2" name="Calibration">
> <file>Calibration.xml</file>
> <sendEventFile>Calibration_nodecoverage_test1.xml       </sendEventFile>
> </state-machine>
> <state-machine id="1" name="Calibration">
> <file>Calibration.xml</file>
> <sendEventFile> 
> Calibration_edgecoverage_test_basispath1.xml</sendEventFile>
> </state-machine>
> </state-machines>
> </vtconfig>
> 
> Here is what I miss:
> 
> - Node coverage: check all states are used
> - Edge coverage: check all transitions are used. I don't understand how 
> edge coverage can fail if node coverage succeed? I think I don't 
> understand what really is edge coverage or how it works...

Edge coverage is different because, as you say, it checks every transition - 
which is essentially the same as confirming every input in IGSTK. Think of it 
this way, if there are two inputs I1 and I2 that transition an IGSTK component 
from state S1 to state S2, then processing either input "covers nodes" S1 and 
S2, but does not validate that both inputs are processed correctly. Only if you 
are in S1 and run different test cases to process I1 and I2 can you be sure your 
component state machine processes them correctly.

Your validation outline is good, it is the common plan for intended use of this 
contributed component.

> - The benefit of using sendEvent files generated from the .scxml is 
> obscure to me compared to generating them from log files (functionnal 
> tests)? It looks like a dog chasing its tail...

Generating sendEvent files from log files is so you can "replay" the state 
machine processing of an actual IGSTK run. Generating them directly from the 
SCXML files lets you create formally defined coverage cases (node, edge, path, 
etc.). This is akin to unit test "white box" coverage, while the replay is like 
tracing through a functional system test after the fact.

> - How does the simulator work? What is the point of defining in the 
> config file twice the same state machine with different sendEventFile 
> (node and edge coverage) ? I though it was meant to run
> with 2 different state machines and confront their states? 

The simulator is designed to simultaneously execute multiple state machines, and 
so the configuration file allows you to identify many state machine and 
sendEvent files. Like IGSTK however, state machine processing is 
single-threaded, meaning at any given point in time the simulator is processing 
a particular input on a particular state machine instance.

The case you mention, of a single state machine with 2 sendEvent files, is a 
little confusing. What is actually going on is there are 2 state machine 
instances but they are both of the same type. For example, consider an IGSTK 
application which uses two tracking devices. Each tracker instance will have a 
corresponding state machine instance, but they will both have the same structure 
(expressed in the SCXML file), say tracker.xml. Each instance can process its 
own sequence of events...

In the
> examples, the rules files are sometimes empty. What is the default 
> behaviour of the simulator if no specific rules are specified?

The rules engine is an experimental component we are integrating with the CDash 
dashboard. Often one may want to just run the simulator and simply observe what 
happens with the state machines. Other times you may want to define test case 
conditions or constraints, which is what the rules are for. It is a bit 
confusing due to its current design, we have a plan to refactor this piece and 
clean it up some so it makes more intuitive sense.

> 
> I know it's a lot of questions... But I'm a bit lost...
> 
> 
> Thanks a lot.
> 
> Regards,
> 
> _______________________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at 
> http://www.kitware.com/opensource/opensource.html
> 
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/cgi-bin/mailman/listinfo/igstk-users

-- 
Kevin A. Gary, Ph.D.
Assistant Professor
Department of Engineering
College of Technology and Innovation
Arizona State University at the Polytechnic Campus
(480)727-1373
http://dcs.asu.edu/faculty/KevinGary
kgary at asu.edu