VTK/Image Stencil Improvements: Difference between revisions

From KitwarePublic
< VTK
Jump to navigationJump to search
 
(12 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Some of the VTK image filters can operate through a "stencil", which is represented by the special data class vtkImageStencilData.  This class stores a binary image mask via run-length encoding.  This encoding is used for the sake of speed and low memory usage.  Stencils have existed in VTK since VTK 4, but are not supported by many filters, and more importantly, were never fully moved over to the VTK 5 pipeline executives.  The goal of this project is correct the shortcomings of the stencils.
The VTK imaging pipeline support the idea of a "stencil," which is a selection region that can have any shape, 2D or 3D.  The stencil is represented by the special data class vtkImageStencilData, which stores a binary image mask via run-length encoding.  Stencils have existed in VTK since VTK 4, but are not supported by many filters, and more importantly, were never fully moved over to the VTK 5 pipeline executives.  The goal of this project is correct the shortcomings.  Ideally, all image filters that have the same output data type as their input data type should support stencilling.


== Description of vtkImageStencilData ==
== Description of vtkImageStencilData ==
Line 6: Line 6:
# Compactness -- image masks are usually very sparse
# Compactness -- image masks are usually very sparse
# Efficiency -- the run lengths can be traversed very fast
# Efficiency -- the run lengths can be traversed very fast
The traversal of the stencil can easily be driven by an iterator (but unfortunately, such an iterator has not been written).  Random-access of voxel values in the stencil is O(''n'') where ''n'' is the number of run-lengths in the row being accessedIt could be made to be O(log(''n'')) but since ''n'' is typically less than four, there would be little advantage in doing so.  
The traversal of the stencil can easily be driven by an iterator, and creating an iterator class is part of this projectThe iterator can be made to be compatible with the vtkImageIterator, which will simplify the conversion of many of the image filters to use stencils.


In the current implementation, stencils are strictly binary, but they could be extended to be multi-value to allow them to be used as labels.  This would require the addition of a scalar array to vtkImageStencilData.
Sequential access of stencil data is done run-by-run, returning contiguous spans just like vtkImageIterator and with similar efficiency.  Random-access is a bit slower, it is O(''n'') where ''n'' is the number of run-lengths in the row being accessed.  It could be made to be O(log(''n'')) but since ''n'' is typically less than four, there would be little advantage in doing so.  In the current implementation, stencils are strictly binary, but they could be extended to be multi-value to allow them to be used as labels.  This would require the addition of a scalar array to vtkImageStencilData.


The superclass of vtkImageStencilData is vtkObjectData, instead of vtkDataSet.  This means that, unlike other kinds of VTK data, a vtkImageStencilData cannot be abstractly represented as a collection of points and cells.
The superclass of vtkImageStencilData is vtkObjectData, instead of vtkDataSet.  This means that, unlike other kinds of VTK data, a vtkImageStencilData cannot be abstractly represented as a collection of points and cells.
Line 32: Line 32:
New classes:
New classes:
* vtkImageStencilAlgorithm - fully upgrade stencils to the VTK 5 pipeline
* vtkImageStencilAlgorithm - fully upgrade stencils to the VTK 5 pipeline
* vtkImageStencilMerge - merge stencils via binary operations
* vtkImageStencilBoolean - combine stencils via logical operations
* vtkROIStencilSource - create stencils of various simple shapes
* vtkROIStencilSource - create stencils of various simple shapes
* vtkLassooStencilSource - create stencils from contours
* vtkLassoStencilSource - create stencils from contours
* vtkImageStencilToImage - convert a stencil into a binary mask image
* vtkImageStencilToImage - convert a stencil into a binary mask image
* vtkImageStencilIterator - an iterator for image stencils
* vtkImageStencilIterator - an iterator for image stencils
Line 45: Line 45:
* Support for label values in stencils
* Support for label values in stencils


== Current Status (Nov 13, 2010) ==
== Current Status (Nov 15, 2010) ==
 
Several classes have been converted to use the new vtkImageStencilIterator:
* vtkImageStencil
* vtkImageBlend
* vtkImageAccumulate
 
There are many VTK-5 fixes left to be done, these will require some changes to vtkImageAlgorithm and its executive, the vtkStreamingDemandDrivenPipeline:
# SetInformationInput needs to become a VTK-5 style input.
# SetStencil needs to become SetStencilInputConnection()
# WholeExtent/UpdateExtent needs special handling in the executive
 
Once these changes are done, it will be easy to convert classes that use vtkImageIterator so that they use vtkImageStencilIterator instead.  This will allow those filters to use stencils.


=== vtkImageStencilAlgorithm ===
=== vtkImageStencilAlgorithm ===


'''Done.'''  But the vtkImageStencilSource class needs some cleanup:
'''Done.'''  But the vtkImageStencilSource class needs some cleanup:
# InformationInput needs to become a VTK-5 style input.
# Default WholeExtent is still an issue, it is inconsistent between subclasses.
# Many filters should be converted to use it.


=== vtkImageStencilMerge ===
=== vtkImageStencilBoolean ===


Not started.
Not started yet.


=== vtkROIStencilSource ===
=== vtkROIStencilSource ===
Line 62: Line 71:
'''Done.'''  Support for boxes, ellipses, and cylinders in X, Y, and Z orientations.
'''Done.'''  Support for boxes, ellipses, and cylinders in X, Y, and Z orientations.


=== vtkLassooStencilSource ===
=== vtkLassoStencilSource ===


'''Done.'''  Support for polygonal and spline contours in X, Y, and Z orientations.  Contours can be stacked, with a different contour for each slice.
'''Done.'''  Support for polygonal and spline contours in X, Y, and Z orientations.  Contours can be stacked, with a different contour for each slice.
=== vtkImageStencilToImage ===
'''Done.'''


=== vtkImageStencilIterator ===
=== vtkImageStencilIterator ===
Line 72: Line 85:
=== vtkPolyDataToImageStencil ===
=== vtkPolyDataToImageStencil ===


'''Done.'''  Supports 2D contours in the XY plane.  Uses new polygon rasterization code in vtkLassooStencilSource.  It could still be made much more efficient and flexible, e.g.:
'''Done.'''  Supports 3D surfaces and 2D contours in the XY plane.  Uses new polygon rasterization code from vtkLassoStencilSource.  It could still be made much more efficient and flexible, e.g.:
# The cutting code could be replaced with faster code from vtkClipClosedSurface.  
# The cutting code could be replaced with faster code from vtkClipClosedSurface.  
# It could support slicing in any direction, like vtkLassooStencilSource.
# It could support slicing in any direction, like vtkLassoStencilSource. '''Done''' as of Sept 13, 2014.

Latest revision as of 16:26, 6 January 2015

The VTK imaging pipeline support the idea of a "stencil," which is a selection region that can have any shape, 2D or 3D. The stencil is represented by the special data class vtkImageStencilData, which stores a binary image mask via run-length encoding. Stencils have existed in VTK since VTK 4, but are not supported by many filters, and more importantly, were never fully moved over to the VTK 5 pipeline executives. The goal of this project is correct the shortcomings. Ideally, all image filters that have the same output data type as their input data type should support stencilling.

Description of vtkImageStencilData

The purpose of vtkImageStencilData is to store a binary image mask. It does so via run-length encoding, where the runs alternate between zero (black) and unity (white). This run-length encoding was chosen for two reasons:

  1. Compactness -- image masks are usually very sparse
  2. Efficiency -- the run lengths can be traversed very fast

The traversal of the stencil can easily be driven by an iterator, and creating an iterator class is part of this project. The iterator can be made to be compatible with the vtkImageIterator, which will simplify the conversion of many of the image filters to use stencils.

Sequential access of stencil data is done run-by-run, returning contiguous spans just like vtkImageIterator and with similar efficiency. Random-access is a bit slower, it is O(n) where n is the number of run-lengths in the row being accessed. It could be made to be O(log(n)) but since n is typically less than four, there would be little advantage in doing so. In the current implementation, stencils are strictly binary, but they could be extended to be multi-value to allow them to be used as labels. This would require the addition of a scalar array to vtkImageStencilData.

The superclass of vtkImageStencilData is vtkObjectData, instead of vtkDataSet. This means that, unlike other kinds of VTK data, a vtkImageStencilData cannot be abstractly represented as a collection of points and cells.

Status as of VTK 5.6

As of VTK 5.6, the support for stencils is unchanged as compared to VTK 4.2. Only a small number of classes can use stencils:

  • vtkImageStencil - combine two images via a stencil
  • vtkImageBlend - apply a stencil while blending images
  • vtkImageReslice - ignore all out-of-stencil voxels, set them to the background color
  • vtkImageAccumulate - generate a histogram of only the voxels in the stencil

Stencils can be created in the following ways:

  • vtkImageToImageStencil - threshold an image to create a stencil
  • vtkImplicitFunctionToImageStencil - generate stencil from an implicit function
  • vtkPolyDataToImageStencil - generate stencil from a surface mesh

The vtkImageStencilData class also has methods for doing in-place binary addition and subtraction of stencils, but there are no filters for performing these operations, hence they cannot be pipelined.

There is currently no way to load stencils from disk or save them to disk, except through conversion to and from vtkImageData. The stencil is stored as a sequence of run-lengths for each row of an image, which is a very compact, but this compactness is lost if the stencil is converted into an image.

Summary of Proposed Improvements

New classes:

  • vtkImageStencilAlgorithm - fully upgrade stencils to the VTK 5 pipeline
  • vtkImageStencilBoolean - combine stencils via logical operations
  • vtkROIStencilSource - create stencils of various simple shapes
  • vtkLassoStencilSource - create stencils from contours
  • vtkImageStencilToImage - convert a stencil into a binary mask image
  • vtkImageStencilIterator - an iterator for image stencils

Improved classes:

  • vtkPolyDataToImageStencil - make it work with 2D contours, instead of just 3D surfaces

There are several other items that would be nice to have, if people can volunteer the time:

  • Stencil support for more filters, e.g. vtkImageMathematics
  • Readers/writers for stencils
  • Support for label values in stencils

Current Status (Nov 15, 2010)

Several classes have been converted to use the new vtkImageStencilIterator:

  • vtkImageStencil
  • vtkImageBlend
  • vtkImageAccumulate

There are many VTK-5 fixes left to be done, these will require some changes to vtkImageAlgorithm and its executive, the vtkStreamingDemandDrivenPipeline:

  1. SetInformationInput needs to become a VTK-5 style input.
  2. SetStencil needs to become SetStencilInputConnection()
  3. WholeExtent/UpdateExtent needs special handling in the executive

Once these changes are done, it will be easy to convert classes that use vtkImageIterator so that they use vtkImageStencilIterator instead. This will allow those filters to use stencils.

vtkImageStencilAlgorithm

Done. But the vtkImageStencilSource class needs some cleanup:

vtkImageStencilBoolean

Not started yet.

vtkROIStencilSource

Done. Support for boxes, ellipses, and cylinders in X, Y, and Z orientations.

vtkLassoStencilSource

Done. Support for polygonal and spline contours in X, Y, and Z orientations. Contours can be stacked, with a different contour for each slice.

vtkImageStencilToImage

Done.

vtkImageStencilIterator

Done. It is not a subclass of vtkImageIterator, though, because vtkImageIterator needs some improvement.

vtkPolyDataToImageStencil

Done. Supports 3D surfaces and 2D contours in the XY plane. Uses new polygon rasterization code from vtkLassoStencilSource. It could still be made much more efficient and flexible, e.g.:

  1. The cutting code could be replaced with faster code from vtkClipClosedSurface.
  2. It could support slicing in any direction, like vtkLassoStencilSource. Done as of Sept 13, 2014.