[Ctk-developers] Scenes

Fri Sep 18 20:00:57 EDT 2009

As I mentioned, Open Inventor supports both types, CGScenes and IPScenes - it is not limited to just the first kind.
This is why it was relatively simple to wrap ITK and VTK in Open Inventor, even though those libraries have nothing to do with CGScenes.

In XIP Builder CGScenes are built top-down using connections in the top/bottom part of the box representing a node, while IPScenes are usually arranged freely on the visual workspace and their connections emanate laterally from the left and right edges of the box. IPScene connections are type-specific and are created by popping up a "field panel" for the object and connecting input-output parameters. Open Inventor supports user-defined field converters so that some connections between fields of different type (eg. text vs. numeric) are converted automatically.

When you start digging in, there is a mountain of details that need to be supported and defining a toolkit-agnostic scene graph may be impossible to attain. That's why I'm an advocate of not re-inventing what others have spent decades refining, and I prefer just creating an XML file definition for toolkits that exist. 

I am beginning to wonder if we are better off defining a common XML file format that contains toolkit-specific declarations. This would save us a lot of time, because we would not have to figure out how to create generic scene descriptions. The file could declare that, in order to parse a particular scene you will need to have, for example, VTK, MITK and IGSTK. The application would make sure those DLLs are loaded, then parse the file and instantiate those objects. We already have to do this in XIP Builder because various objects are organized in separate libraries, and those libraries need to be present at runtime otherwise certain scene graphs cannot be fully loaded (objects from missing libraries are shown as red boxes).

If we go with this approach then the focus in CTK would shift towards implementing "bridge" scene objects that allow connections between different toolkits - objects that would accept an input from an XIP object and provide a VTK output, and so on. As long as CTK provides a library of such objects and a file format to define how to create scenes that intermix the use of all out toolkits, our goal for interoperability would be met.

-----Original Message-----
From: Stephen Aylward [mailto:stephen.aylward at kitware.com] 
Sent: Friday, September 18, 2009 7:01 PM
To: Paladini, Gianluca (SCR US)
Cc: Wolf Ivo; ctk-developers at commontk.org
Subject: Re: [Ctk-developers] Scenes

Hi,

Gianluca and Ivo bring up a good point..."what is a scene?".   The
wiki begins with that discussion and a list of requirements, and it seems as though we might need to hash it out a bit more.

To being, perhaps we need to distinguish computer graphics scene graphs ("CGScenes") from image processing scene graphs ("IPScenes").
The two serve different purposes.

CGScenes are used in two of the largest, community imaging
environments: XIP (OpenInventor) and Slicer (MRML).   CGScenes use
traversal-based semantics to define the context at a node, this is
motivated by openGL's use of contexts during rendering.   CGScenes are
highly developed and have wonderful utility in computer graphics, and clearly they are a solid basis for imaging applications.

IPScenes have also been developed independently by several groups (MITK, IGSTK, itkSpatialObjects, and OpenMAF).  An IPScene is a tree of transforms, where the transform at a node only influences its children.  That is, you don't traverse the scene graph to understand
the context of the image stored at a node.   IGScenes have
demonstrated themselves to be useful for representing the relative location of patients and instruments for surgical guidance systems, the interactions of physiological systems for gate/motion analysis and prosthesis design, the multiscale representation of biological processes for multi-system disease modeling, and numerous other
application areas.   Sorry if my bias is showing here.

As Gianluca mentioned, we need to save the state of an application as
it processes its data.   CGScenes make this easy.  The centralization
of GUI and model information simplifies saving state and simplifies
the implementation of undo/redo capabilities.   However, it could be
argued that such mingling does confound the exchange of data across different applications and does create massive scenes that mix physical and GUI constructs.  Defining a common API for such disparate types of data does perhaps necessitate the over simplification of the
API.   For example, the scene requirements listed on our wiki suggest
that we need to be able to ask for the physical location of an
object...how can you ask a GUI button for its physical location?   How
can you apply a 3D transform to a slider?   The notion of saving the
state of an application is critical, but perhaps it is better suited for a data structure other than the scene graph; one that has its own, specific API.

So, we need both: a scene of objects and a record of the GUI state.
On some implementations, it may be useful to combine them as
facilitated by CGScenes.   However, we should also allow other
implementations to keep them separate.

In particular, perhaps the XIP can be used as a reference
implementation demonstrating their combined storage.    Additionally,
it would be interesting to see if MITK, IGSTK, or others can be adopted as a companion implementation that demonstrates the utility of keeping GUI and physical objects in separate data structures.

Let's begin the talk on Friday with "What is a scene"   - I fear we
won't get beyond this question unless we have already had a few beers,
and it will be a bit early in the day.   So perhaps we should try to
limit the time for each topic so that we can hit some potentially fertile topics for discussion as well as the more cantankerous topics.

Thanks,
Stephen

On Fri, Sep 18, 2009 at 5:56 PM, Paladini, Gianluca (SCR US) <gianluca.paladini at siemens.com> wrote:
> The advantage of a scene graph comes primarily from those nodes which 
> define attributes rather than drawn objects, because those attributes 
> can be inherited in lower branches of the scene and represent the 
> state of the system at any point during scene graph traversal - that 
> is a key feature, especially when it comes to transform matrices. "Separator"
> objects prevent those state changes from propagating, while "Group"
> objects don't. It is this mechanism that makes scene graphs ideal for 
> setting up coordinate systems and controlling parts that move with 
> respect to each other.
> Without that it's not a scene graph, it's a pipeline. Pipelines are 
> supported in Open Inventor too, but via objects called engines rather 
> than nodes.
>
> -----Original Message-----
> From: ctk-developers-bounces at commontk.org
> [mailto:ctk-developers-bounces at commontk.org] On Behalf Of Wolf Ivo
> Sent: Friday, September 18, 2009 5:02 PM
> To: Stephen Aylward; ctk-developers at commontk.org
> Subject: Re: [Ctk-developers] Scenes
>
> Hi Stephen,
> As for the MITK team, we traditionally like the type of definition of 
> a scene that you gave and the mode-view-controller pattern. Both is 
> pretty close to what is used within MITK (except for some extensions, 
> which would be great to have) and therefore could nicely be integrated 
> from CTK. Nevertheless, my feeling is that starting with a scene 
> implementation is not the easiest way to go.
>
> And caution (correct me, if I am wrong): The Open Inventor definition 
> of a scene is different from yours (and ours)! Nodes in an Open 
> Inventor scene graph are not necessarily 'objects' (called shape nodes 
> there), but can also be lights, materials (e.g. color), transforms 
> (matrices), switches etc. Thus, the definition "a scene is composed of 
> objects that fill a portion of space and time" is not compatible with 
> Open Inventor (a material node does not fill a portion of space). 
> Maybe it is possible (and desirable) to combine both ideas of defining 
> a scene graph: the more data-centered approach defined above and the 
> visualization-centered approach of Open Inventor. But we should be 
> careful that we are talking about the same things (a topic for the list of questions).
>
> Regarding a brief presentation, we recently added some semantic 
> information to our scene description, which might be useful for ctk, 
> too.
>
> Best regards,
>
> Ivo
>
>
>
> ________________________________
>
> Von: ctk-developers-bounces at commontk.org im Auftrag von Stephen 
> Aylward
> Gesendet: Do 17.09.2009 02:18
> An: ctk-developers at commontk.org
> Betreff: [Ctk-developers] Scenes
>
>
>
> Hi,
>
> ctkScene:  Love it?  Hate it?  Know you can make it better?
>
> Regardless of your current attitude, please send me your ideas and 
> presentations!!!
>
> I'd like to help organize the discussion on ctkScene that is scheduled 
> for next Friday.
>
> 1) If you are interested in making a brief presentation (less than 10 
> minutes), then please send me an email with the title.   I will add it 
> into our agenda.
>
> 2) If you have revisions to the Wiki page on ctkScene - please make 
> those revisions by 5pm EST (22:00 GMT) on Friday.   I am going to 
> summarize those wiki pages as a set of slides at the start of the 
> meeting.
>      http://my-trac.assembla.com/protoctk/wiki/ctkScene
>
> 3) If you have questions, concerns, needs, wants, or rants regarding 
> the ctkScene, please send them to me and I will create a set of slides 
> that list "open-issues" to help focus our discussions.   We only have 
> one-hour, so I'd like to focus on high-level issues that remain, prior 
> to getting into implementation details.   For example, should a 
> ctkScene be only a file format or should it also be implemented as a 
> set of C++ classes?   Should a ctkScene be specific to or independent 
> of the visualization library?   Should a ctkScene be our primary data 
> structure, or should our primary data structure simply be an image?
> ...please add to this list of questions...
>
> Thanks!
> Stephen
>
> --
> Stephen R. Aylward, Ph.D.
> Director of Medical Imaging
> Kitware, Inc. - North Carolina Office
> http://www.kitware.com <http://www.kitware.com/> stephen.aylward 
> (Skype)
> (919) 969-6990 x300
> _______________________________________________
> Ctk-developers mailing list
> Ctk-developers at commontk.org
> http://public.kitware.com/cgi-bin/mailman/listinfo/ctk-developers
>
>
> _______________________________________________
> Ctk-developers mailing list
> Ctk-developers at commontk.org
> http://public.kitware.com/cgi-bin/mailman/listinfo/ctk-developers
>

--
Stephen R. Aylward, Ph.D.
Director of Medical Imaging
Kitware, Inc. - North Carolina Office
http://www.kitware.com
stephen.aylward (Skype)
(919) 969-6990 x300