[Insight-users] Memory Allocation Error in Java

Luis Ibanez luis.ibanez at kitware.com
Mon Dec 20 17:19:52 EST 2004


Hi Evert,

Thanks for your report.

There is however something unclear from your description.

You say that you tried "both solutions", I'm assuming that
one of them was the addition of the "SetReleaseDataFlag()",

is that right ?




 From the remaining of your description, it seems that the
filter that is taking most of the memory is the ShapeDetection
level set filter.


Your ShapeDetection filter is instantiated in 3D for float
images.  Your input image has 3276800 pixels, and encoded
in 4 bytes/ pixel the size will go to


                     12.5 MegaBytes


Shape detection uses

   - Speed image
   - InitialLevelSet
   - FeatureImage


Shape detection doesn't use the Advection images, but even
if it allocates (which it shouldn't) that should be about

     12.5M * 3


all that will add up to something like

          12.5 M * 6 = 75 Mb

which is far from the 2Gb limit that you are hitting.




Here is an important question:

   are you getting the exception just at the beginning
of the processing ?  or do you get it after some iterations
of the level set filter have been executed ?


If you are in the second case, this may indicate a memory
leak bug...

One easy way to look at it is to use the memory reporting
of your system (e.g. ksysguard on Linux KDE, or the Task Manager
on Windows) and see how the memory allocation is growing as the
program is execution.

If you find a linear and progressive increase in the memory,
then that may be a sign that a memory leak is going on.


Please run your example again for the small image where
you still succeed to run whithout the exception being
thrown, monitor the memory allocation and let us know
what you find.


    Thanks


       Luis


------------------------
Evert van Velsen wrote:

> Hi Luis,
> 
> I've tried both solutions, and it worked for my 128x128x200 tiff stack,
> both with and without the ShapeDetection filter.
> But after that I tried it for another tiff stack of 128x128x300 and the
> same memory allocation error occured. Even when I omitted the
> ShapeDetection filter, and only the FastMarching filter was used.
> Hope you can do something with this,
> 
> Evert
> 
> 
>>Hi Evert
>>
>>
>>Please try the following:
>>
>>
>>
>>1) Temporarily remove the ShapeDetection filter from
>>    your code and check if the stages leading up to
>>    the FastMarching filter are still challenging the
>>    memory resources.
>>
>>    This will help us to locate where the memory is
>>    being consumed.
>>
>>
>>
>>
>>2) Try invoking in all the filters the methods
>>
>>
>>         SetReleaseDataFlag( true );
>>
>>
>>    This flags forces a filter to release its output
>>    memory when it no longer needed by the downstream
>>    pipeline.
>>
>>    So far, every filter in the pipeline is holding the
>>    memory of its output image, therefore accumulating
>>    more and more memory. Your images however, don't seem
>>    to be large enough for justifying a 2GB comsumption.
>>
>>
>>
>>
>>Please let us know what you find.
>>
>>
>>
>>    Thanks
>>
>>
>>
>>       Luis
>>
>>
>>
>>-----------------------
>>Evert van Velsen wrote:
>>
>>
>>>Hi Luis and others,
>>>
>>>I used a tiff-stack of 128x128 with 200 slices with a 8-bit grayscale.
>>>The
>>>file is output of ImageJ.
>>>Furthermore I will add my code of the ShapeDetectionLevelSet, so you can
>>>see what I'm doing, and maybee see what's wrong with it.
>>>
>>>Thanks,
>>>
>>>Evert
>>>------------------------------Java Code----------------------------
>>>itkBinaryThresholdImageFilterF3US3_Pointer thresholder =
>>>itkBinaryThresholdImageFilterF3US3.itkBinaryThresholdImageFilterF3US3_New();
>>>        thresholder.SetLowerThreshold( -1000.0f );
>>>  		thresholder.SetUpperThreshold(     0.0f );
>>>  		thresholder.SetOutsideValue(  0  );
>>>  		thresholder.SetInsideValue(  255 );
>>>
>>>  		itkImageFileReaderF3_Pointer reader =
>>>itkImageFileReaderF3.itkImageFileReaderF3_New();
>>>		itkImageFileWriterUS3_Pointer writer =
>>>itkImageFileWriterUS3.itkImageFileWriterUS3_New();
>>>	    reader.SetFileName(path);  //locatation of the input file
>>>		writer.SetFileName(directory+nameU); //output location
>>>
>>>        itkCurvatureAnisotropicDiffusionImageFilterF3F3_Pointer
>>>smoothing
>>>=
>>>itkCurvatureAnisotropicDiffusionImageFilterF3F3.itkCurvatureAnisotropicDiffusionImageFilterF3F3_New();
>>>        itkGradientMagnitudeRecursiveGaussianImageFilterF3F3_Pointer
>>>gradientMagnitude =
>>>itkGradientMagnitudeRecursiveGaussianImageFilterF3F3.itkGradientMagnitudeRecursiveGaussianImageFilterF3F3_New();
>>>	    itkSigmoidImageFilterF3F3_Pointer sigmoid =
>>>itkSigmoidImageFilterF3F3.itkSigmoidImageFilterF3F3_New();
>>>
>>>        sigmoid.SetOutputMinimum(  0.0f  );
>>>  		sigmoid.SetOutputMaximum(  1.0f  );
>>>
>>>  		itkFastMarchingImageFilterF3F3_Pointer fastMarching =
>>>itkFastMarchingImageFilterF3F3.itkFastMarchingImageFilterF3F3_New();
>>>  		itkShapeDetectionLevelSetImageFilterF3F3_Pointer shapeDetection =
>>>itkShapeDetectionLevelSetImageFilterF3F3.itkShapeDetectionLevelSetImageFilterF3F3_New();
>>>  		//pipeline:
>>>  		smoothing.SetInput( reader.GetOutput() );
>>>		gradientMagnitude.SetInput( smoothing.GetOutput() );
>>>		sigmoid.SetInput( gradientMagnitude.GetOutput() );
>>>		fastMarching.SetInput(sigmoid.GetOutput());
>>>		shapeDetection.SetInput( fastMarching.GetOutput() );
>>>		shapeDetection.SetFeatureImage( sigmoid.GetOutput() );
>>>		thresholder.SetInput( shapeDetection.GetOutput() );
>>>		writer.SetInput( thresholder.GetOutput() );
>>>
>>>  		smoothing.SetTimeStep( 0.059 );
>>>  		smoothing.SetNumberOfIterations(  5);
>>>  		smoothing.SetConductanceParameter( 9 );
>>>
>>>        double sigma = 1.0;
>>>		double alpha =  -0.5;
>>>		double beta  =  1.5;
>>>		gradientMagnitude.SetSigma(  sigma  );
>>>		sigmoid.SetAlpha( alpha );
>>>		sigmoid.SetBeta(  beta  );
>>>
>>>	   	itkNodeContainerF3_Pointer seeds =
>>>itkNodeContainerF3.itkNodeContainerF3_New();
>>>		itkIndex3 seedPosition = new itkIndex3();
>>>		itkLevelSetNodeF3 node = new itkLevelSetNodeF3();
>>>		float seedValue = -10f;
>>>
>>>                seedPosition.SetElement(0,61); //x
>>>		seedPosition.SetElement(1,81); //y
>>>		seedPosition.SetElement(2,4);  //z
>>>		node.SetValue( seedValue );
>>>		node.SetIndex( seedPosition);
>>>	        seeds.Initialize();
>>>		seeds.InsertElement(0,node);
>>>
>>>  		fastMarching.SetTrialPoints( seeds.GetPointer()  );
>>>  		fastMarching.SetSpeedConstant( 1.0 );
>>>  		fastMarching.SetOutputSize(reader.GetOutput().GetBufferedRegion().GetSize()
>>>);
>>>
>>>  		float curvatureScaling   = 2.5f;
>>>  		float propagationScaling = 2.5f;
>>>
>>>        shapeDetection.SetPropagationScaling(  propagationScaling );
>>>  	shapeDetection.SetCurvatureScaling( curvatureScaling );
>>>        shapeDetection.SetMaximumRMSError( 0.02 );
>>>  	shapeDetection.SetNumberOfIterations( 800 );
>>>
>>>  		try
>>>	    {
>>>	    	writer.Update();
>>>	    }
>>>	  	catch(Exception e)
>>>	    {
>>>	    	NJ.write("Error: "+ e);
>>>	    }
>>>---------------end of code----------------------------
>>>
>>>
>>>
>>>
>>>>Hi Evert,
>>>>
>>>>Thanks for trying the command line options for java.
>>>>
>>>>
>>>>
>>>>It is now time for "Plan B".
>>>>
>>>>
>>>>
>>>>The question to answer is:
>>>>
>>>>     What are you doing in your pipeline that
>>>>     is taking so much memory (more than 1 Gb).  ?
>>>>
>>>>
>>>>
>>>>Please give us a list of:
>>>>
>>>>
>>>>- The filters that you connected in your pipeline,
>>>>
>>>>- The size (in pixels) of the input image,
>>>>
>>>>- The pixel type that you are using for instantiating
>>>>  each one of the intermediate filters.
>>>>
>>>>
>>>>
>>>>One thing that you may want to try is to enable the
>>>>ReleaseDataFlag() in all the filters in your pipeline.
>>>>This flag allows you to release memory of the filter
>>>>as their Update() progress in the pipeline. In this
>>>>way you discard memory that is no longer needed.
>>>>
>>>>
>>>>Please let us know about the information in the list
>>>>above.
>>>>
>>>>
>>>>   Thanks
>>>>
>>>>
>>>>
>>>>      Luis
>>>>
>>>>
>>>>
>>>>-----------------------
>>>>Evert van Velsen wrote:
>>>>
>>>>
>>>>
>>>>>Hi Luis,
>>>>>
>>>>>the fist two options you proposed, didn't help unfortunately. The
>>>>>memmory
>>>>>usage was already at it max; even 1GB :S
>>>>>Also I tried to adapt line 186 into the following:
>>>>>throw MemoryAllocationError(__FILE__, __LINE__, "Failed to allocate
>>>>>memory
>>>>>for image: "+size,"ImportImageContainer::AllocateElements");
>>>>>But the whole program crashed.
>>>>>
>>>>>Do you have any other solutions to see how much memory had to be
>>>>>allocated?
>>>>>Otherwise I had to sepparate my ImageStack into subsamples I think.
>>>>>
>>>>>Thanks,
>>>>>
>>>>>Evert
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Hi Evert,
>>>>>>
>>>>>>The Java virtual machine has a natural limitation to the amount
>>>>>>of memory made available its processes.
>>>>>>
>>>>>> http://java.sun.com/docs/hotspot/gc5.0/ergo5.html
>>>>>>
>>>>>>You may want to first attempt to increase the memory made available
>>>>>>to your program by the virtual machine.
>>>>>>
>>>>>>http://java.sun.com/j2se/1.5.0/docs/tooldocs/solaris/java.html
>>>>>>
>>>>>>This can be done with the command line options for the "java"
>>>>>>command:
>>>>>>
>>>>>>"   -Xmsn
>>>>>>
>>>>>>   Specify the initial size, in bytes, of the memory allocation pool.
>>>>>>This value must be a multiple of 1024 greater than 1MB. Append the
>>>>>>letter k or K to indicate kilobytes, or m or M to indicate megabytes.
>>>>>>The default value is 2MB. Examples:
>>>>>>
>>>>>>      -Xms6291456
>>>>>>      -Xms6144k
>>>>>>      -Xms6m
>>>>>>
>>>>>>
>>>>>>-Xmxn
>>>>>>   Specify the maximum size, in bytes, of the memory allocation pool.
>>>>>>This value must a multiple of 1024 greater than 2MB. Append the letter
>>>>>>k
>>>>>>or K to indicate kilobytes, or m or M to indicate megabytes. The
>>>>>>default
>>>>>>value is 64MB. Examples:
>>>>>>
>>>>>>      -Xmx83886080
>>>>>>      -Xmx81920k
>>>>>>      -Xmx80m
>>>>>>
>>>>>>"
>>>>>>
>>>>>>
>>>>>>
>>>>>>If you still find memory allocation problems, you may want to add a
>>>>>>message to the file:
>>>>>>
>>>>>>       Insight/Code/Common/itkImportImageContainer.txx
>>>>>>
>>>>>>in line 186 in order to specify how much memory is trying to allocate.
>>>>>>
>>>>>>It is likely that some of the signed/unsigned convertions from Java to
>>>>>>C++ is creating a situation where an obscene amount of memory is being
>>>>>>requested for an image.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>Please let us know what you find.
>>>>>>
>>>>>>
>>>>>>  Thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>>      Luis
>>>>>>
>>>>>>
>>>>>>
>>>>>>------------------------
>>>>>>Evert van Velsen wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Hi there,
>>>>>>>
>>>>>>>when running my java-written ITK program, which I've combined with
>>>>>>>ImageJ,
>>>>>>>I received the following error:
>>>>>>>Error: java.lang.RuntimeException:
>>>>>>>c:\insighttoolkit-1.8.1\code\common\itkImportImageContainer.txx:186:Failed
>>>>>>>to allocate memory for image.
>>>>>>>
>>>>>>>The input of the program is a 200 slice 3d-image (tiff-format) of
>>>>>>>128x128,
>>>>>>>and the error occured when performing the
>>>>>>>ShapeDetectionLevelSetFilter
>>>>>>>with 200 seedpoints (one on each slice).
>>>>>>>I'm using ITK 1.8.1 with java 1.4.2_05
>>>>>>>
>>>>>>>Does anyone know how I can fix this problem?
>>>>>>>
>>>>>>>Thanks,
>>>>>>>
>>>>>>>Evert
>>>>>>>_______________________________________________
>>>>>>>Insight-users mailing list
>>>>>>>Insight-users at itk.org
>>>>>>>http://www.itk.org/mailman/listinfo/insight-users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>Insight-users mailing list
>>>Insight-users at itk.org
>>>http://www.itk.org/mailman/listinfo/insight-users
>>>
>>>
>>
>>
>>
>>
>>
> 
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org
> http://www.itk.org/mailman/listinfo/insight-users
> 
> 






More information about the Insight-users mailing list