[Ctk-developers] CTK Scene

Sun Aug 23 22:10:25 UTC 2009

Hi Gianluca,

Good point!   I am all for re-using code, and so I think we need to
decide on a library, but perhaps not build one ourselves.  Here is why
I think we should share code for a scene, and not just file formats:

1) Widgets and display (application building): The QtWidgets being
discussed by Steve, for example, include display widgets.  Those are
going to require us to define how images are stored in memory.
Typically display widgets must handle entire scenes - they must handle
the transforms between objects, handle different display methods for
different objects (some as surfaces and some as volume rendering), etc
etc.   Also, CTK can enable application building only if its
components can interoperate.   We should, in my opinion, avoid trying
to duplicate QtWidgets for a multitude of different scene libraries.

2) Algorithms: I tend to design algorithms that operate on scenes:
registration for image guided surgery, vessel network extraction and
characterization, longitudinal studies, as well as statistical atlas
formation methods.     These algorithms typically require full
knowledge of the pose of objects in space and time, with respect to
parent objects, and with respect to one another, i.e., a scene.

3) Impact: it seems like if we stop at a file format, then we miss the
opportunity to re-use code and to enable the community.   Promoting
code re-use was a driving force in my work at UNC and now at Kitware
because I believe that by making algorithms and data containers
open-source, then we can have a greater impact on the field than 99.9%
of the journal articles published.   Coding is often viewed as an evil
necessity in medical image research - but what we need to realize that
source code is actually the only language in which our methods and
ideas can be truly communicated.  Distributing algorithms as code
exposes our methods' parameters, assumptions, limitations, and
real-world utility and enables others to rapidly build upon our work -
journal articles typically do not).

But - I do see advantages to openAPI instead of openSource in certain
situations.   In particular, when we develop a community around CTK,
we should encourage even the non-openSource folks to participate.  For
example, we should avoid all GPL code that would require all source to
be open.   It would also be nice if we allowed for some methods to be
distributed as binary-only.   Or perhaps methods having non-commercial
licenses.   However, the core should remain BSD.

My 2 cents (~0.01 euro).

Stephen

On Fri, Aug 21, 2009 at 12:21 PM, Paladini, Gianluca (SCR
US)<gianluca.paladini at siemens.com> wrote:
> Hi Marco,
> I can see this concept of Actors corresponding to Open Inventor Manipulators, and the concept of RenderWindows corresponding to Open Inventor Examiners.
>
> Just playing devil's advocate - are we trying to reinvent the wheel?
>
> The goal of CTK is to provide a common set of functionality that we can all use in our respective frameworks. The idea of having a common, generalized scene graph file format that we can use to exchange information across toolkits can be very useful, but is it useful to implement a new scene graph library, knowing that there are so many off-the-shelf libraries developers can already use?
>
> For example in the case of XIP there is one free, open source version of Open Inventor from Sgi which we use in OpenXIP and two commercial versions from Mercury (http://www.vsg3d.com) and SystemsInMotion (http://www.coin3d.org) which we use to implement XIP-based products. Alternatively there are free open source libraries like OpenSG (http://opensg.vrsource.org/trac), OpenSceneGraph (http://www.openscenegraph.org/projects/osg), OpenRM (http://www.openrm.org), NVSG (http://developer.nvidia.com/object/nvsg_home.html).
>
> Since existing open source scene graph initiatives have put several years of development into their libraries, an effective way to design CTK's scene graph interchangeable file format may be to look at what exists already - a 'competitive analysis' if you will. Do you think this could be a useful exercise? I admit we only look at Open Inventor for our XML file format, therefore we may be missing support for functionality existing in other scene graph libraries.
>
>  - Gianluca
>
> -----Original Message-----
> From: ctk-developers-bounces at commontk.org [mailto:ctk-developers-bounces at commontk.org] On Behalf Of Stephen Aylward
> Sent: Friday, August 21, 2009 8:20 AM
> To: Marco Viceconti
> Cc: ctk-developers at commontk.org
> Subject: Re: [Ctk-developers] CTK Scene
>
> Both options have merit.
>
> I agree that exporting single objects should be supported.   It allows
> the system to interact with other (legacy) systems and, as you said,
> provides IO flexibility.    One question regarding terminology...when
> you say "Actors" you are not referring to vtkActors, right?   I'm
> fairly certain you are not, but I mention it because this is a point
> of contention for me at times...I've found vtkActors to have too
> limited an API to be used as scene objects (e.g., no standard method
> exists for querying if a point is inside of them, limited concept of
> parent-child relationships... - they fulfill almost none of the
> requirements that I posted on the wiki:
> http://my-trac.assembla.com/protoctk/wiki/ctkScene).
>
> Perhaps RenderWindow-specific information can be broadcast via traits.
>  Traits can be used to describe a RenderWindow (3D vs 2D, MIP
> transfer function, Grayscale vs Color, Interactive vs passive) and for
> these traits each object in the scene may optionally change its
> default appearance (For ObjectX, if renderWindow is 2D,  grayscale,
> and interactive; then display ObjectX as a semi-transparent contour,
> and show its control points so that the contour can be edited).   Any
> application can override these defaults with application-specific
> behaviour, but those settings should (perhaps?) not be stored in the
> scene if those settings are not inherent to the object.  The reason
> the renderWindow is given traits instead of the object having traits
> is so that the application developer (who is defining the renderWindow
> within his/her application) can specify the general nature of how
> objects will appear within it so as to be most effective for the user
> / task.
>
> Really interesting idea about exposing the renderwindow.   I need to
> learn/think more about that - not certain how it promotes the exchange
> of information between multiple views or between views and logic
> modules within a single application.
>
> This discussion reminds me of yet another approach...a scene rendering
> pipeline developed by Julien Jomier called SpatialObject Viewers.   In
> it, each renderWindow had an objectFactory associated with it.  This
> objectFactor would generate a different renderer for each object fed
> to it, based on the object's type.  The application developer could
> choose different objectFactories (one that produced MIPs, one that was
> interactive, one that was 3D) for each renderWindow in the application
> and in that way achieve a desired behaviour/appearance.   Julien
> presented this system to the NAMIC group years ago:
>   http://www.na-mic.org/Wiki/images/f/.../Na-Mic-SLC05-SpatialObjects.ppt
> Wow, we even have a website for them (this is a surprise to me -
> google is amazing):
>   http://public.kitware.com/SOViewer/
> This design pattern could be extended so that the objects, by
> specifying how they should be viewed in renderWindows having
> particular traits, could influence which renderer the renderWindow's
> objectFactory produced for them.  Furthermore, there is no need to
> extend a renderWindow or application as each new object type is
> defined.  Only objectFactories need to be extended, and those are
> intended to be extended via shared libraries.
>
> My thoughts for the morning...
>
> Stephen
>
>
>
> On Fri, Aug 21, 2009 at 2:58 AM, Marco Viceconti<viceconti at tecno.ior.it> wrote:
>> One of the goal of MAF3 is to fully separate VMEs (our data objects) from
>> Views and from Operations, so as to maximise reusability, and to make
>> possible that each of these MAF resources can be added at run time as a
>> plugin.
>>
>> This of course raise the debate on how to provide a neutral representation
>> of the visualisation components. Firstly, Stephen implies that the right
>> point where to cut is the scene, as most of the visualisation libraries do.
>>  But I would like to bring some alternative ideas on the table:
>>
>> a) Actors.  One could image to export single actors instead of whole scene.
>>  this would reduce the granularity, and would make conceptually easier to
>> develop distributed applications where single actors are being
>> generated/updated by separate processes, and the visualisation application
>> has the responsibility to compose the scene with them and to render it.
>> Actually this idea is not necessarily in conflict with the one Stephen and
>> Gianluca advocate; we could image an extensibility mechanism to the scene
>> format, so that I can also export a single actor using the same XML syntax.
>>
>> b)  RenderWindows: to the other extreme, in MAF we are considering to expose
>> out of the visualisation module not the scene, but the RenderWindow.  The
>> rationale behind this is that the true conceptualisation nowadays is in
>> choosing for each type of actor a certain rendering style, so that
>> multi-actors Views produce a coherent representation. So for example, in our
>> MIP View volumes are rendered with a MIP visual pipe, but surfaces are
>> rendered with a flat surface rendering, so that it appears consistent with a
>> radiography-like image where geometric objects are usually man-made and thus
>> of homogenous opacity.  In this perspective the View becomes like a virtual
>> imaging machine, which produces a visual representation for each digital
>> object according to the type and properties of this object in a coherent
>> way.
>>
>> The problem with approach b) is that I have difficulties to see how this
>> could be made compatible with scene approach.  One possibility could be to
>> extend the XML syntax to include an optional section where one can also
>> specify the rendering elements; does this make any sense?
>>
>> I do not have a strong opinion on this, but I thought it might be useful we
>> try to thing a bit out of the box here.
>>
>> Cheers
>>
>> Marco
>>
>>
>>
>> Il giorno 21/ago/09, alle ore 00:39, Stephen Aylward ha scritto:
>>
>>> Continuing the discussion of CTK data types...
>>>
>>> I propose that we have a ctkScene as a driving datatype.
>>>
>>> First, perhaps, we need to agree on the definition of a scene.   I
>>> propose that a ctkScene should have two main traits:
>>>
>>> 1) Generically, a scene is composed of objects that fill a portion of
>>> space and time
>>> * Objects have a position in space
>>> * Objects have an orientation in space
>>> * Objects have a range in time
>>> * Given a point in physical space and time, it should be possible to
>>> determine if an object contains it or not.
>>> * An object should contain at least one point in space and time
>>> * The concept of units is paramount in a scene.
>>> ** Units describe the spatial and chronological scale of an object
>>>
>>> 2) A scene represents a hierarchy of object in space and time.
>>> * Objects have a single parent (except a root object) and may have one
>>> or more children objects
>>> ** That is, a scene is a tree
>>> * Moving a parent object in space or time should (by default) cause
>>> its children to experience the same movement in space and time
>>> * The coordinate system for root objects is defined as "real-world"
>>> space.   It is an absolute space.
>>> * A scene may have multiple roots
>>> ** It is not necessary to represent everything in space from a single
>>> root.
>>> ** One or more roots are used to contain the physical/temporal objects
>>> relevant to a particular task.
>>> * Objects implicitly include their child objects
>>> ** When querying the time/space extent of an object (e.g., to
>>> determine if a point is inside of it or to determine its bounding box)
>>> that query typically also includes its child objects in its
>>> computations.
>>> * Most queries to and responses from objects should be with respect to
>>> real-world space (i.e., the coordinate system of the root object).
>>> ** For example, querying an object if it contains a physical point is
>>> done using real-world coordinates (i.e., a coordinate system common to
>>> all objects in its tree).
>>>
>>> I'm adding this to the wiki at:
>>> http://my-trac.assembla.com/protoctk/wiki/ctkScene
>>>
>>> Stephen
>>>
>>> --
>>> Stephen R. Aylward, Ph.D.
>>> Director of Medical Imaging
>>> Kitware, Inc. - North Carolina Office
>>> http://www.kitware.com
>>> stephen.aylward (Skype)
>>> (919) 969-6990 x300
>>> _______________________________________________
>>> Ctk-developers mailing list
>>> Ctk-developers at commontk.org
>>> http://public.kitware.com/cgi-bin/mailman/listinfo/ctk-developers
>>>
>>
>> --------------------------------------------------
>> MARCO VICECONTI, PhD                             (viceconti at tecno.ior.it)
>> Laboratorio di Tecnologia Medica              tel.   39-051-6366865
>> Istituto Ortopedico Rizzoli                            fax.   39-051-6366863
>> via di Barbiano 1/10, 40136 - Bologna, Italy
>>
>> Tiger! Tiger! Burning bright in the forest of the night,
>> what immortal hand or eye could frame thy fearful symmetry?
>> --------------------------------------------------
>> Opinions expressed here do not necessarily reflect those of my employer
>>
>>
>>
>> _______________________________________________
>> Ctk-developers mailing list
>> Ctk-developers at commontk.org
>> http://public.kitware.com/cgi-bin/mailman/listinfo/ctk-developers
>>
>
>
>
> --
> Stephen R. Aylward, Ph.D.
> Director of Medical Imaging
> Kitware, Inc. - North Carolina Office
> http://www.kitware.com
> stephen.aylward (Skype)
> (919) 969-6990 x300
> _______________________________________________
> Ctk-developers mailing list
> Ctk-developers at commontk.org
> http://public.kitware.com/cgi-bin/mailman/listinfo/ctk-developers
>

-- 
Stephen R. Aylward, Ph.D.
Director of Medical Imaging
Kitware, Inc. - North Carolina Office
http://www.kitware.com
stephen.aylward (Skype)
(919) 969-6990 x300