[vtk-developers] vtkImageConvolve crashes when malloc fails

[Mathieu Coursolle]

Hi VTK developers,

I am performing a series of filters, where some of them are  
vtkImageConvolve.

My input vtkImageData is usually close to 100Mbytes, and the execution  
of my pipeline
seems to cause some memory fragmentation. Therefore, malloc will  
eventually fail, causing
a crash in vtkImageConvolve.

If malloc fails during the execution of a pipeline, then the  
scalarPointer of the resulting vtkImageData is NULL. If a connected  
vtkImageConvolve filter receives such a vtkImageData
as its input, it tries to access the scalar pointer (which is NULL),  
causing a crash.

A solution would be to check the pointers:

   // ADDED CODE BEGIN--------
   // Make sure the scalar pointers are valid.
   if (!inData[0][0]->GetScalarPointer() || !outData[0]- 
 >GetScalarPointer())
     {
	vtkErrorMacro(<< "Execute: input and/or output image does not have  
any scalar value.");
     return;
     }
   // ADDED CODE END--------

   switch (inData[0][0]->GetScalarType())
     {
     vtkTemplateMacro(
       vtkImageConvolveExecute(this, inData[0][0],
                               static_cast<VTK_TT *>(inPtr), outData[0],
                               static_cast<VTK_TT *>(outPtr),
                               outExt, id, inInfo));

     default:
       vtkErrorMacro(<< "Execute: Unknown ScalarType");
       return;
     }

So I was wondering,
1) Is that the right way to avoid such crashes?
2) I guess that problem might occur with other filters as well...

Thanks.

Mathieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20080603/fc06e7a0/attachment.html>