[Insight-developers] Image memory model

Sat May 24 01:28:28 EDT 2008

Thanks for engaging in this discussion with me.

To cut to the chase, my actual problem is politics -- the "not
invented here syndrome". I am trying to introduce ITK into my project,
but have encountered resistance from other developers. The memory
model is their biggest concern; they seem blinded to the plethora of
advantages ITK offers. As such I am investigating the possibility of
removing this stumbling block. I guess suggesting a breaking change to
ITK due to the internal politics at my work is probably not a very
good idea :P Let me describe the problem further to see if there is
another way round.

Some background. We have an extensive image processing application
which uses a slice-contiguous model (based on DICOM). Images are read
from disk, loaded into memory, image processing operations performed,
and results rendered to screen or written back to disk. There may be 5
or more 1GB images open at a time (many situations require less
memory, some require more). To integrate ITK I am currently 1.
importing each slice using the Import filter, copying the slices into
contiguous memory using the Tile filter, applying the desired ITK
algorithm(s), and converting the contiguous memory back to slices (if
required). The conversions consume unnecessary time and (more
importantly) memory, which is not acceptable in various circumstances
(having to consume an extra 5GB of memory to duplicate image slices is
not very nice). Some of the ITK operations I want to use in this
setting include (but are not limited to): registration, level set
segmentation, intensity-based segmentation, distance transforms, and
mesh operations (image to mesh, mesh to image).

So, how exactly would the proposed itk::SliceContiguousImage subclass
work? I guess I would need to override the PixelContainer type (by
adding a new template argument to itk:Image, which defaults to
ImportImageContainer) and override the Set/GetPixel(..) methods. Then
I guess I would need to create a new ImportSlicedImagePixelContainer
to replace ImportImageContainer. I'm not sure I understand how the new
image type would interact with linear iterators -- these iterators use
m_Buffer+m_Offset throughout. Could I somehow override
ImageBase::ComputeOffsetTable(..) to return the correct offset based
on the slice memory model? Also I have no idea where to start with
neighborhood iterators. Readers/writers are not a concern for me, but
I know the Wiki proposal page mentions them for sake of completeness.

FYI here is a quick survey of some of the filters I would like to use
with a slice contiguous memory model:
 # Registration: cost functions compute gradient using
RecursiveSeparableFilter, which copies memory using linear iterator
 # Level set: uses neighborhood iterators,
SparseFieldCityBlockNeighborList uses stride table
 # Fast Marching: uses linear iterator and Get/SetPixel(..)
 # Morphology: uses neighborhood iterators
 # Distance transforms: uses morphology filter, and linear iterator
 # Image to mesh: uses linear iterators
 # Mesh to image: uses linear iterator

As I see it my options are:
1. Follow my current path and duplicate memory (and fight harder for
ITK inclusion)
-OR-
2. attempt to get linear *and* neighbor iterators to work with a slice
contiguous memory model.

As discussed option 1 is not very appealing. Option 2 seems like a
possibility (although I'm not sure level set filters extending from
SparseFieldLevelSetImageFilter will work). Can you help me flesh out
the details regarding this option?

Thanks again for having this discussion with me.

Regards, Dan

2008/5/23 Luis Ibanez <luis.ibanez at kitware.com>:
>
> Hi Dan,
>
> Thanks for this detailed description of the options.
>
> Is there a particular set of algorithms that are
> of interest for the slice-contiguous image ?
>
> The reason for the question is that, at the end, most ITK filters
> access images via iterators, and therefore, a modification in the
> iterators behavior may be enough for making a large number of filters
> compatible with slice-contiguous images.
>
> In addition, since most filters are templaged over the image type,
> your suggestion of creating a new itk::SliceContiguousImage is perfectly
> feasible. (e.g. similat to what was done with the itk::OrientedImage).
>
> The Registration Framework will also be very easy to convert too, since
> images there are only accessed via the image iterators, and through the
> image interpolators via GetIndex() methods.
>
> We may be able to find a middle ground where, without converting the
> entire toolkit, a large number of algorithms can be made compatible with
> a slice-base memory arrangement and probably with other alternate memory
> strategies (for example: octrees).
>
>
>   Regards,
>
>
>
>        Luis
>
>
> -------------------
> Dan Mueller wrote:
>>
>> Hi Insight-Developers,
>>
>> I am investigating the feasibility of enhancing the ITK image memory
>> model to support a slice-contiguous approach.
>>
>> I notice this topic has been on the radar for a number of years:
>>
>>  http://www.itk.org/Wiki/ITK_Roadmap_2005_2006#Slice_dis-contiguous_images
>>    http://www.itk.org/Wiki/Proposals:Slice_contiguous_images
>>
>>> From where I sit, the biggest hurdle for this change is not the
>>
>> implementation itself, but ITK's backwards compatibility policy:
>>    http://www.itk.org/Wiki/ITK_Backward_Compatibility_Open_Discussion
>>
>> I am interested in soliciting developer feedback on possible steps
>> forward. Here are some questions which might help generate further
>> discussion:
>>  # 1. How strong is the need in the (medical) imaging community for
>> this feature?
>>  # 2. What are the possible mechanisms for introducing this feature
>> without breaking backwards compatibility (too much)?
>>  # 3. Should a sparse memory model also be investigated/introduced
>> during this process?
>>
>> My thoughts:
>> # 1. How strong is the need in the (medical) imaging community for this
>> feature?
>> The advent of 64-bit computing has certainly reduced this need (it is
>> now possible to have multiple copies of a large image in memory),
>> however good memory management practices should still be followed
>> (this is especially important in commercial environments). I am
>> working on a (commercial) project at the moment in which a DICOM image
>> (using a slice-contiguous memory model) is open in memory and I want
>> to import this into ITK; currently the only option is to duplicate the
>> memory for ITK's contiguous memory model (which is not very
>> desirable). For me at least, a slice-contiguous memory model is very
>> desirable.
>>
>> # 2. What are the possible mechanisms for introducing this feature
>> without breaking backwards compatibility?
>> The two current proposals are to either (a) template the itk::Image
>> class over the PixelContainer or (b) use inheritance/virtual functions
>> to allow the existing itk::Image to operate on a variety of
>> PixelContainer classes. Both of these proposals will require the
>> removal of  Image::GetBufferPointer(), a massively breaking change.
>> With the current backwards compatibility policy I'm not sure this is
>> possible. Perhaps one option is to mark this function as deprecated
>> and leave it as such for a suitable time (6-12 months?). The problem
>> certainly is one of balance and moderation: do we strictly adhere to
>> the backwards compatibility policy at the cost of not being able to
>> make (potentially important) changes/additions to the toolkit? (Ross'
>> and Simon's position statements on the backward compatibility open
>> discussion wiki page certainly argue for good balance). What is the
>> best way to achieve a good balance in this case?
>>
>> # 3. Should a sparse memory model also be investigated/introduced
>> during this process?
>> I think adding the slice-contiguous feature is enough to start with.
>> Obviously the sparse memory model should be in the back of our minds
>> if/when slice-contiguous support is added.
>>
>> Thanks for any input on this discussion.
>>
>> Regards, Dan Mueller
>> dan dot muel at gmail dot com
>> _______________________________________________
>> Insight-developers mailing list
>> Insight-developers at itk.org
>> http://www.itk.org/mailman/listinfo/insight-developers
>>
>