[vtk-developers] Better prop3D for displaying images

[David Gobbi]

Hi All,

I'd like to present an idea that I've had for a long while, but have
never had time to implement.  There have always been lots of questions
and complaints on the list about how difficult it is to get VTK to
display image slices properly.  The vtkImageActor has made things a
bit easier, but it is sorely lacking in functionality compared to the
venerable vtkVolume and vtkActor classes.

Here's my proposal:  add a "vtkImage" class to VTK that has the
following features:

1) The slice would not be limited just to a "ZSlice" like vtkImageActor.

1a) The slice plane would be defined with a vtkPlane or some similar method.

1b) The bounds of the displayed image would be set automatically to
the full extent of the image intersected by the plane.  The lack of
this functionality has always been a huge stumbling block for VTK
users.

1c) The plane would be set in mapper coordinates.

1d) A method could also be provided to set the slice plane in world
coordinates, so that the prop position/orientation controls what slice
is displayed.


2) It will have a separate mapper object, just like vtkVolume does.
I've been writing some generic picking code, and the fact that
vtkImageActor has no mapper is a real pain.  It means that some of the
good old code for doing cell/point picking just doesn't work with
ImageActor.  Also, a separate mapper means that different rendering
techniques can be used, similar to vtkVolume.

2a) The mapper will allow clipping planes to be associated with the data.

2b) The default mapper will just display slices via 2D textures.

2c) A 3D texture mapper can be used to accelerate slicing.

2d) A projection-based mapper could be used to generate radiographs
(DRRs) from the volume.

2e) There could be a mapper that would allow arbitrary fragment
shaders to be added, or perhaps this could be done in the property.


3) It will have a separate property object.

3a) The property is where the lookup table will be set.

3b) Also, Opacity.

3c) Also, window/level.  The window/level mapping would be applied
before that data is passed to the lookup table, so if window/level are
used, the range of the lookup table should be [0,1].  This would allow
the same lookup table to be shared between images that are displayed
with different window/level settings.

3d) A Shade boolean to control whether direction lighting is used in
the render, along with Ambient, Diffuse, and other lighting
parameters.  The vtkImageActor does not allow control over these
parameters.


4) The vtkCellPicker and vtkPointPicker will work efficiently and
generically with it.

4a) The vtkPointPicker will return both a valid PointId and IJK point index.

4b) the vtkCellPicker will return a CellId, IJK cell index, and PCoords.


The reasoning behind all of this is that images have always been
second-class citizens in VTK.  It seems to me that most of the people
contributing image processing code to VTK over the past ten years,
including myself, haven't been studying VTK's generic data interface
thoroughly enough.  As a result, image processing & display seems kind
of tacked-on.  A proper image rendering infrastructure for VTK would
go a long way to fix that, and would stop people from continually
reinventing the wheel.

Unfortunately I don't have the resources to do more than a fraction of
the work myself, but if there are people out there who can contribute,
I'm more than willing to provide my expertise.

    David