[vtk-developers] vtkImageConvolve crashes when malloc fails

[mcoursolle at rogue-research.com]

I guess this is a common problem when dealing with large data sets
(100Mbytes). I can reproduce the problem easily just by calling a series
of malloc and free. The memory allocation is always bellow 150 MBytes in
my app, but there is so many fragmentation that malloc fails. If I call
malloc, I can detect such a failure and take appropriate measure (saving
data, etc.).

However, if malloc is called inside VTK, I'd like the pipeline execution
to stop before it reaches a crash.

Mathieu

> The parallel computing experts should address this, but basically the
> solution to large data is to stream the data in pieces. While this is not
> always possible, VTK provides good support for this especially for image
> data. This means that your source object must be able to respond to data
> requests that are less than the whole extent of the image.
>
> Also, make sure that you are setting the release data flags, etc. to
> discard
> memory in the pipeline when it is no longer needed.
>
> W
>
> On Tue, Jun 3, 2008 at 4:37 PM, Berk Geveci <berk.geveci at kitware.com>
> wrote:
>
>> If memory allocation fails, you will run into a crash sooner or later
>> unfortunately. Even if you were to add such a check, somewhere down
>> the pipeline some other class is going to die. IMHO, the only
>> "graceful" way of handling this is to throw an exception that the
>> application catches. Then it would have to report an error and shut
>> down. Adding bunch of if statements for an exceptional case is
>> probably not a good idea.
>>
>> -berk
>>
>> 2008/6/3 Mathieu Coursolle <mcoursolle at rogue-research.com>:
>> > Hi VTK developers,
>> > I am performing a series of filters, where some of them are
>> > vtkImageConvolve.
>> > My input vtkImageData is usually close to 100Mbytes, and the execution
>> of
>> my
>> > pipeline
>> > seems to cause some memory fragmentation. Therefore, malloc will
>> eventually
>> > fail, causing
>> > a crash in vtkImageConvolve.
>> > If malloc fails during the execution of a pipeline, then the
>> scalarPointer
>> > of the resulting vtkImageData is NULL. If a connected vtkImageConvolve
>> > filter receives such a vtkImageData
>> > as its input, it tries to access the scalar pointer (which is NULL),
>> causing
>> > a crash.
>> > A solution would be to check the pointers:
>> >   // ADDED CODE BEGIN--------
>> >   // Make sure the scalar pointers are valid.
>> >   if (!inData[0][0]->GetScalarPointer() ||
>> !outData[0]->GetScalarPointer())
>> >     {
>> > vtkErrorMacro(<< "Execute: input and/or output image does not have any
>> > scalar value.");
>> >     return;
>> >     }
>> >   // ADDED CODE END--------
>> >
>> >
>> >
>> >   switch (inData[0][0]->GetScalarType())
>> >     {
>> >     vtkTemplateMacro(
>> >       vtkImageConvolveExecute(this, inData[0][0],
>> >                               static_cast<VTK_TT *>(inPtr),
>> outData[0],
>> >                               static_cast<VTK_TT *>(outPtr),
>> >                               outExt, id, inInfo));
>> >     default:
>> >       vtkErrorMacro(<< "Execute: Unknown ScalarType");
>> >       return;
>> >     }
>> > So I was wondering,
>> > 1) Is that the right way to avoid such crashes?
>> > 2) I guess that problem might occur with other filters as well...
>> > Thanks.
>> > Mathieu
>> >
>> > _______________________________________________
>> > vtk-developers mailing list
>> > vtk-developers at vtk.org
>> > http://www.vtk.org/mailman/listinfo/vtk-developers
>> >
>> >
>> _______________________________________________
>> vtk-developers mailing list
>> vtk-developers at vtk.org
>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>
>
>
>
> --
> William J. Schroeder, PhD
> Kitware, Inc.
> 28 Corporate Drive
> Clifton Park, NY 12065
> will.schroeder at kitware.com
> http://www.kitware.com
> 518-371-3971 (phone and fax)
>