[Insight-developers] Improving support for color in ITK

[Dan Mueller]

Hi ITK developers,

There are segmentation tasks which require different image color
spaces. As I need to perform (supervised statistical) classification
on color images (digitized pathology images), I intend to improve
support for non-RGB color spaces in ITK. My ultimate goal is a
submission to the Insight Journal consisting of various
classes/filters for converting to/from various color spaces. However,
before I go too far, I would appreciate some thoughts/suggestions from
more experienced ITK developers.

I started my task by taking an inventory of current color
functionality within ITK:

RGBPixel
NumericTraitsRGBPixel
* Basic data container for (linear) RGB pixels

RedPixelAccessor
GreenPixelAccessor
BluePixelAccessor
NthElementImageAdaptor
VectorIndexSelectionCastImageFilter
* Provide access to individual (RGB) color components

RGBToVectorImageAdaptor
RGBToVectorPixelAccessor
* Provide access to RGB pixel as vector pixel

ConvertPixelBuffer
RGBToLuminanceImageFilter
* Provide conversions from (linear) RGB to luminance

Obviously there is a lack of support for other common color spaces
such as HSV, XYZ, Lab.

I also looked at VTK, which contains the following functions for color
space conversion (see Common/vtkMath.h):
    RGBToHSV
    HSVToRGB
    LabToXYZ
    XYZToLab
    XYZToRGB
    RGBToXYZ
    RGBToLab
    LabToRGB

I noticed vtkMath::RGBToXYZ(..) and vtkMath::XYZToRGB(..) have some
code regarding conversion from linear RGB to non-linear R'G'B' (ie.
sRGB). However this code is commented out, citing incompatible with
OpenGL.

I then read Charles Poynton's Color and Gamma FAQs which are
referenced in IO/itkConvertPixelBuffer.h:
    http://www.poynton.com/PDFs/ColorFAQ.pdf
    http://www.poynton.com/PDFs/GammaFAQ.pdf
<aside>The URLs in itkConvertPixelBuffer.h are outdated and broken.</aside>

I am still processing these documents, but the author seems to suggest
HSB (hue, saturation, brightness) and HLS (hue, lightness, saturation)
"are useless for conveyance of accurate color information" because the
"formulations are based on RGB components whose chromaticities and
white point are unspecified".

All this has left me somewhat confused. So I guess my big question is:
should I stay with the "simple" approach taken by VTK, which --
although not the whole truth -- seems adequate? Or should I aim for a
more "complete" solution, with support for non-linear color spaces
(eg. sRGB), white point specification, etc?

Brad: From a recent post to the users list, you seem knowledgeable on
this topic. Perhaps you have some advice?
     http://www.itk.org/pipermail/insight-users/2009-July/031394.html

Without further advice I will implement the VTK color conversions in a
base filter which accepts VectorImage types. That is, rather than
create a new pixel container for each of the color spaces, the three
channels can simply reside in a vector. For example:

typedef unsigned char BasePixelType;
typedef itk::RGBPixel< BasePixelType > RgbPixelType;
typedef itk::Image< RgbPixelType, 2 > RgbImageType;
typedef itk::ImageFileReader< RgbImageType>  RgbReaderType;
ReaderType::Pointer reader = ReaderType::New();
reader->SetFileName( "D:/Temp/color1.png" );

typedef itk::RGBToVectorImageAdaptor< BasePixelType > AdapterType;
AdapterType::Pointer imageRgb = AdapterType::New();
imageRgb->SetImage( reader->GetOutput() );

typedef itk::VectorImage< BasePixelType, 2 > VectorImageType;
typedef itk::ConvertColorSpaceImageFilter< BasePixelType, 2 > ConvertFilterType;
ConvertFilterType::Pointer converter = ConvertFilterType::New();
converter->SetInput( imageRgb );
converter->SetInputColorSpace( ColorSpace::RGB );
converter->SetOutputColorSpace( ColorSpace::CIELAB );
converter->Update( );
VectorImageType::Pointer imageCielab = converter->GetOutput( );
VectorImageType::IndexType index;
index.Fill( 10 );
BasePixelType lightness = imageCielab->GetPixel( index ).GetElement( 0 );

The advantage of using vectors to hold the color images is that no new
pixel types (or traits) need to be added/maintained. The disadvantage
is that the user now needs to be careful which color components are
currently held by the vector. For example pixel.GetElement( 0 ) may
return R(GB), L*(a*b*), X(YZ), or H(SV). Do you feel this is too
confusing?

Sorry for the long email. Thanks for any advice.

Regards, Dan