[Insight-users] Evolution : overall pipeline memory print

[Julien Michel]

Dear ITK users,

In our toolkit based on ITK, we are processing remote sensing images
which are potentially huge. We are taking advantages of the ITK natural
streaming capabilities, along with our own file writer able to write
parts of images (depending on the image format).

Unfortunately, streaming is often hard to set up in an optimal way :
dividing images into too small pieces leads to intensive HD reading and
writing, while too large pieces leads to memory allocation error. The
question we are trying to answer is "what is the optimal size of the
streaming strip for a given input image and pipeline ?" This is hard to
tell because it depends on the number of filters in the pipeline, as
well as the number of outputs of each of these filters.

 From a user point of view, we would like to be able to call a
GetPipelineMemoryPrint() method on the last filter of the pipeline,
which will return the overall amount of memory that the pipeline would
use if the Update() method is triggered on this filter. This would be
quite simple to implement. Here are the evolution we thought of to
support this new functionality.

- in itk::DataObject

Add the following method:

virtual std::streamsize GetMemoryPrint(void)
{
   return 0;
}

- in subclasses of itk::DataObject:

If necessary, override the GetMemoryPrint() method.

For instance, in itk::Image, the GetMemoryPrint() implementation would be:
virtual std::streamsize GetMemoryPrint(void)
{
   return
static_cast<std::streamsize>(this->GetRequestedRegion().GetNumberOfPixels()
	*sizeof(PixelType);
}
Another example for the itk::VectorImage class :
virtual std::streamsize GetMemoryPrint(void)
{
   return
static_cast<std::streamsize>(this->GetRequestedRegion().GetNumberOfPixels()
	*getNumberOfComponentsPerPixel()*sizeof(InternalPixelType);
}
Of course we can also add proper implementation for other DataObject, if
necessary.

- in itk::ProcessObject

Add the following method:

std::streamsize GetPipelineMemoryPrint(void)
{
std::streamsize size = 0;
for(unsigned int idx = 0; idx<this->GetNumberOfOutputs();++idx)
{
	size+=this->GetOutput(idx)->GetMemoryPrint();
}
for(unsigned int idx = 0; idx<this->GetNumberOfInputs();++idx)
{
	if(this->GetInput(idx)->GetSource())
	{
	size+=
	this->GetInput(idx)->GetSource()->GetPipelineMemoryPrint();
	}
}

We do not know if such a feature is of interest for ITK users. Anyway,
this is not an heavy evolution and does not compromise backward
compability. But since it takes place in very high level classes, we
would be interested to contribute these evolutions to ITK rather than
evolving our own version of these ITK classes. This would ensure our
compatibility with any up-to-date version of ITK, not only our own. Of
course we would write and test the code, and submit these evolutions to
ITK-Journal, but before doing this we would like to have your feedback
on this proposition. Would ITK be interested in ? Is the proposed
solution likely to work ?

Thanks a lot for having patiently read this long email to the end,

Best regards,

Julien MICHEL