ITK Release 4/Image Class Hierarchy Refactoring

Image Class Hierarchy Refactoring

Motivation

The goal of this proposal is to refactor ITK's image class hierarchy to make adding other image types to ITK easier and to simplify the interface of image classes wherever possible.

ITK currently supports several types of image:

• Image
• VectorImage
• SpecialCoordinatesImage
  • PhasedArray3DSpecialCoordinatesImage
• BloxImage
• SparseImage
• LabelMap
• ImageAdaptors and subclasses

Most of the image types supported by ITK assume that the image topology is an n-dimensional lattice and that voxels have regular physical spacing in each dimension. While it makes sense to preserve the topology assumption, the regular spacing assumption should be relaxed wherever possible. The regular spacing assumption covers many cases in biomedical imaging, but it may hinder ITK's application to certain types of images. For example, certain motorized stages used in confocal microscopes fail to achieve regular spacing in z when acquiring a 3D image. Helpfully, these stages report the positions of z-planes acquired during image stack collection. Assuming a regular z-spacing in place of the actual z-plane positions may result in significant errors in image analysis algorithms run on the image.

The PhasedArray3DSpecialCoordinatesImage class is an example of an existing image type in ITK that does not assume regular sample spacing. Nevertheless, its interface still has the SetSpacing() and GetSpacing() methods defined in the ImageBase class (and overriden in SpecialCoordinatesImage). These methods are required for compatibility in filter pipelines, but their presence in this class is misleading. Other types of images may consist of samples not associated with a physical space. For these image types, class methods for setting metadata about the physical embedding of the image are not needed.

Goals

The primary goal in this refactoring is to remove methods in image types where they are not needed.

Other goals include:

• Add a RectilinearImage class - physically embedded image with origin, irregular spacing in each dimension, and orientation
• Preserve the current interface of Image for backwards compatibility

Requirements

• Make changes fully backwards compatible with examples and tests (all examples and tests should compile and run without modification and all tests should pass)
• Each of the physically embedded image types will properly calculate transforms from indices to physical coordinates and back.
• All physically embedded images should support orientations.

Challenges and Potential Pitfalls

• ImageAdaptors should present the correct interface for the image type they adapt. This can be accomplished with partial template specialization of the ImageAdaptor class to avoid modifying any of the existing adaptors.
• Some filters exploit regular spacing of images to achieve high performance and will need to be modified to handle the new image types or throw an exception when they encounter an unsupported type:
  • [Fill these in as they are encountered]
• Filters that require voxel positions should always use methods TransformIndexToPhysicalPoint, etc.
• Will interpolators work out-of-the-box for Images with implementations of TransformIndexToPhysicalPoint, etc?

Design

All image types will be subclasses of ImageBase, as they are now. ImageBase will be stripped to the bare minimum functionality required to support images with lattice topology. Subclasses will provide support for additional metadata about the images.

Class Hierarchy

The class hierarchy will look like this:

• ImageBase< unsigned int VDimension >
  • LabelMap< class TPixel, unsigned int VDimension >
  • PhysicalImageBase< unsigned int VDimension >
    • RegularImageBase< unsigned int VDimension >
      • RegularImage< class TPixel, unsigned int VDimension >
      • Image< class TPixel, unsigned int VDimension > (aliased to RegularImage)
        • BloxImage< class TPixel, unsigned int VDimension >
        • SparseImage< class TNode, unsigned int VDimension >
      • VectorRegularImage< class TPixel, unsigned int VDimension >
      • VectorImage< class TPixel, unsigned int VDimension > (aliased to VectorRegularImage)
    • RectilinearImageBase < unsigned int VDimension >
      • RectilinearImage <class TPixel, unsigned in VDimension >
    • PhasedArray3DSpecialCoordinatesImage< class TPixel >

All Base classes specify the interface for only the metadata of the image. This design is required to support the method CopyInformation(). For example, you may encounter a situation where you copy information from a RegularImage<float, 3> to a RegularImage<double, 3>. Dynamic casting from a RegularImage<float, 3> to a RegularImage<double, 3> won't work, so the method will throw an exception. Casting a RegularImage<float, 3> to a RegularImageBase< 3 > will work, enabling access to the metadata methods (GetSpacing(), GetOrigin(), etc.) from the source image.

Specific methods defined or overridden in each class are provided below.

Image Types

ImageBase

Methods:

• Allocate()
• ComputeIndex()
• ComputeOffset()
• ComputeOffsetTable()
• CopyInformation()
• GetBufferedRegion()
• GetImageDimension()
• GetLargestPossibleRegion()
• GetNumberOfComponentsPerPixel()
• GetOffsetTable()
• GetRequestedRegion()
• Graft()
• Initialize()
• InitializeBufferedRegion()
• RequestedRegionIsOutsideOfTheBufferedRegion()
• SetBufferedRegion()
• SetLargestPossibleRegion()
• SetNumberOfComponentsPerPixel()
• SetRequestedRegion(const RegionType &region)
• SetRequestedRegion(DataObject *data)
• SetRequestedRegionToLargestPossibleRegion()
• UpdateOutputData()
• UpdateOutputInformation()

PhysicalImageBase

Methods overridden from ImageBase

• CopyInformation()

Additional methods

• GetOrigin()
• SetOrigin()
• GetDirection()
• SetDirection()

RegularImageBase

Methods overridden from PhysicalImageBase

• CopyInformation()

Additional methods

• ComputeIndexToPhysicalPointMatrices()
• GetSpacing()
• SetSpacing()
• TransformContinuousIndexToPhysicalPoint(const ContinuousIndex< TCoordRep, VImageDimension > &index, Point< TCoordRep, VImageDimension > &point) const
• TransformIndexToPhysicalPoint(const IndexType &index, Point< TCoordRep, VImageDimension > &point) const
• TransformLocalVectorToPhysicalVector(const FixedArray< TCoordRep, VImageDimension > &inputGradient, FixedArray< TCoordRep, VImageDimension > &outputGradient) const
• TransformPhysicalPointToContinuousIndex(const Point< TCoordRep, VImageDimension > &point, ContinuousIndex< TCoordRep, VImageDimension > &index) const
• TransformPhysicalPointToIndex(const Point< TCoordRep, VImageDimension > &point, IndexType &index) const

Image (subclass of itkRegularImageBase)

Methods overridden from RegularImageBase

• Allocate()
• Graft()
• Initialize()

Additional methods

• []
• FillBuffer()
• GetBufferPointer()
• GetNeighborhoodAccessor()
• GetPixel(const IndexType &index) const
• GetPixel(const IndexType &index)
• GetPixelAccessor(void)
• GetPixelAccessor(void) const
• GetPixelContainer()
• GetPixelContainer() const
• SetNeighborhoodAccessor()
• SetPixel(const IndexType &index, const TPixel &value)
• SetPixelContainer(PixelContainer *container)

RectilinearImageBase

Methods overridden from PhysicalImageBase

• CopyInformation()
• SetLargestPossibleRegion()

Additional methods

• ComputeSpacingPrefixSum()
• GetDefaultSpacing()
• GetDimensionSpacing(unsigned int dimension)
• InitializeWithDefaultSpacings()
• SetDefaultSpacing()
• SetSpacing(unsigned int dimension, long index, Array<PointValueType>)
• TransformContinuousIndexToPhysicalPoint(const ContinuousIndex< TCoordRep, VImageDimension > &index, Point< TCoordRep, VImageDimension > &point) const
• TransformIndexToPhysicalPoint(const IndexType &index, Point< TCoordRep, VImageDimension > &point) const
• TransformLocalVectorToPhysicalVector(const FixedArray< TCoordRep, VImageDimension > &inputGradient, FixedArray< TCoordRep, VImageDimension > &outputGradient) const
• TransformPhysicalPointToContinuousIndex(const Point< TCoordRep, VImageDimension > &point, ContinuousIndex< TCoordRep, VImageDimension > &index) const
• TransformPhysicalPointToIndex(const Point< TCoordRep, VImageDimension > &point, IndexType &index) const

RectilinearImage

Methods overridden from RectilinearImageBase

• Allocate()
• Graft()
• Initialize()

Additional methods

• []
• FillBuffer()
• GetBufferPointer()
• GetPixel(const IndexType &index) const
• GetPixel(const IndexType &index)
• GetPixelAccessor(void)
• GetPixelAccessor(void) const
• GetPixelContainer()
• GetPixelContainer() const
• GetNeighborhoodAccessor()
• GetNeighborhoodAccessor() const
• SetPixel(const IndexType &index, const TPixel &value)
• SetPixelContainer(PixelContainer *container)

LabelMap

No change needed, but the new interface will not have the methods:

• GetDirection()
• GetSpacing()
• SetDirection()
• SetSpacing()

If LabelMaps have a physical embedding, then the LabelMap class should be changed to inherit from PhysicalImageBase.

BloxImage

This class currently descends from Image. If it has no spatial embedding, then it could be changed to descend from a new DigitalImage class.

SparseImage

This class currently descends from Image. If it has no spatial embedding, then it could be changed to descend from a new DigitalImage class.

ImageAdaptors

We should be able to define the ImageAdaptor class to present the proper API for each of the new image types. This requires partial template specialization of the ImageAdaptor template class to maintain backwards compatibility. Specifically, several definitions of the ImageAdaptor class will need to be defined, one for each image type.

To work with the CopyInformation() method in the various Image subclasses, each ImageBase type (RegularImageBase, RectilinearImageBase, etc.) will need to have a corresponding ImageAdaptor type (RegularImageAdaptor, RectilinearImageAdaptor, etc.), each of which is a subclass of its corresponding image base type. They CANNOT be subclasses of the image types with pixel containers because of the problem with dynamic casting in the CopyInformation() method mentioned above.

ResampleImageFilter and WarpImageFilter

The method SetOutputParametersFromImage() takes and ImageBase as input. Now that the metadata is stored in subclasses of ImageBase, this may be suboptimal.

One solution is to try to dynamic_cast the ImageBase input to the various descendants of ImageBase (PhysicalImageBase, RegularImageBase, RectilinearImageBase, etc.). Upon successful casting, the relevant parameters from the ImageBase input can be copied over to the filter. However, say we're resampling to a RectilinearImage with the ResampleImageFilter. The ResampleImageFilter would need to have a method to support individual voxel spacings. But this method isn't needed for resampling to an Image, so it "goes to waste."

Alternatively, these classes could be specialized for the output image type much like the ImageAdaptor is specialized.

Implementation Plan

After all phases of implementation, existing tests must compile and pass.

Phase 1

• Implement PhysicalImageBase< unsigned int VDimension >
• Implement RegularImageBase
• Make Image and VectorImage subclasses of RegularImageBase
• Make SpecialCoordinatesImage a subclass of RegularImageBase

Example implementation complete

Phase 2

• Write RegularImageAdaptor
• Specialize the ImageAdaptor template for Image, VectorImage, and PhasedArray3DSpecialCoordinatesImage classes.
• Remove unnecessary methods from adaptor classes.
• Remove unnecessary methods, typedefs, and member variables from ImageBase, PhysicalImageBase, and RegularImageBase.

Example implementation complete

Phase 3

• Implement RectilinearImageBase and RectilinearImage
• Specialize the ImageAdaptor template for RectilinearImages
• Add tests for rectilinear images

Example implementation complete

Participants

• Cory Quammen (UNC Chapel Hill)