[vtk-developers] vtk java memory problem

[Sebastien Jourdain]

I forgot to say that your application can be easily multi-threaded to
improve the performances regardless the fact that VTK is not
thread-safe...

On Wed, Dec 29, 2010 at 5:22 PM, Sebastien Jourdain
<sebastien.jourdain at kitware.com> wrote:
> Hi Mark,
>
> If you want to make sure no underneath pipeline connection is kept
> through your dataset that you keep inside your HashSet, you should use
> a deepcopy like in the following lines.
>
>  final vtkImageData binaryImage = stencil.GetOutput();
>  final vtkImageData imageToKeep = new vtkImageData();
>  imageToKeep.DeepCopy(binaryImage);
>  binaryImages.add(imageToKeep);
>
> If you want some internal optimization on the Java GC side, you can
> declare all your variables inside the loop "final".
>
> For the VTK GC side, we didn't set it automatically for thread safety
> reasons. But you are doing the right thing by calling
> "vtkGlobalJavaHash.GC()".
> On the other hand System.gc() and System.runFinalization() are not
> mandatory and you should pre-allocate your set like that to prevent
> reallocation...
>
>   int size = 100;
>   HashSet<vtkImageData> binaryImages = new HashSet<vtkImageData>(size);
>   for (int i=0; i<size; i++) {
>       final vtkImageData blankImage = new vtkImageData();
>       ...
>    }
>
> Hope this will solved your memory issue,
>
> Seb
>
>
> On Tue, Dec 28, 2010 at 10:54 PM, Mark Roden <mmroden at gmail.com> wrote:
>> Hi all,
>>
>> I'm trying to track down some memory handling problems when wrapping
>> VTK 5.6.1 with Java.  These problems become very severe very quickly
>> for what we're trying to do.  I've provided some sample code that
>> isolates the issue while still being fundamentally true to the goals
>> of the software.  The long and the short of it is that there appear to
>> be three different numbers for memory:
>> 1) the memory that Java thinks it has
>> 2) the memory that we can calculate java should have (not the same as #1)
>> 3) the actual memory the process is using (far more than #2).
>>
>> The application allows for manual delineations of regions of interest
>> using contours.  The contours are then extruded and stenciled, and
>> somewhere along the line, it looks like memory is leaking.  This is on
>> windows 7 x64 using vtk 5.6.1.
>>
>> To demonstrate the memory problem, this application performs the
>> following steps:
>> 1) creates a HashSet of vtkImageData
>> 2) loops for i to N
>> 3) for each loop, create a vtkImageData filled with an extruded and
>> stenciled sphere.  Put that image into the HashSet from step 1.
>>
>> Some of the current pointers for VTK use with Java is to make sure to
>> avoid the Set(Get) paradigm, which this code does.
>>
>> On my machine, the Java runtime thinks it has 183 mb of memory
>> allocated, regardless of whether or not the vtkImageData are persisted
>> from one iteration to the next (ie, whether or not they are added to
>> the HashSet).  The calculated size of the HashSet grows linearly.  The
>> actual amount of memory as reported by the Task Manager, however,
>> grows exponentially.  Don't run this code unless you can handle 4gb of
>> allocation in memory, otherwise killing the process can be painful.
>>
>> Obviously, if you remove the garbage collection calls, the memory
>> amounts will be different, but then the app doesn't get very far at
>> all.  We've resigned ourselves to manual garbage collection calls;
>> given the size of our allocations, that just appears to be the way of
>> the world.  The problem is that even with those manual garbage
>> collection calls, memory appears to be leaking still.
>>
>> Potential culprits here:
>> A leak in some portion of the extrusion/stenciling chain
>> A leak in garbage collection
>> Placing the image into the HashSet is causing multiple copies that
>> aren't being tracked properly
>> ???
>>
>> How do I fix this issue?
>>
>> Thanks,
>> Mark
>>
>>
>>
>> Here is the code:
>>
>> import vtk.*;
>> import java.util.HashSet;
>>
>> public class VTKMemoryError {
>>
>>    static {
>>        System.loadLibrary("vtkCommonJava");
>>        System.loadLibrary("vtkFilteringJava");
>>        System.loadLibrary("vtkGenericFilteringJava");
>>        System.loadLibrary("vtkGraphicsJava");
>>        System.loadLibrary("vtkHybridJava");
>>        System.loadLibrary("vtkImagingJava");
>>        System.loadLibrary("vtkIOJava");
>>        System.loadLibrary("vtkRenderingJava");
>>        System.loadLibrary("vtkVolumeRenderingJava");
>>        System.loadLibrary("vtkWidgetsJava");
>>    }
>>    public static void main(String[] args) {
>>        HashSet<vtkImageData> binaryImages = new HashSet<vtkImageData>();
>>        for (int i=0; i<100; i++) {
>>            vtkImageData blankImage = new vtkImageData();
>>            int[] extent = {0, 511, 0, 511, 0, 74};
>>            double[] origin = {0, 0, 0};
>>            double[] spacing = {1, 1, 3};
>>            blankImage.SetExtent(extent);
>>            blankImage.SetOrigin(origin);
>>            blankImage.SetSpacing(spacing);
>>            blankImage.SetScalarTypeToUnsignedChar();
>>            blankImage.AllocateScalars();
>>            double[] bounds = blankImage.GetBounds();
>>
>>            vtkSphereSource sphere = new vtkSphereSource();
>>            sphere.SetRadius(50);
>>            sphere.SetCenter((bounds[0]+bounds[1])/2.0,
>> (bounds[2]+bounds[3])/2.0, (bounds[4]+bounds[5])/2.0);
>>            sphere.Update();
>>            vtkPolyData sphereData = sphere.GetOutput();
>>
>>            vtkLinearExtrusionFilter extruder = new vtkLinearExtrusionFilter();
>>            extruder.SetInput(sphereData);
>>            extruder.SetVector(0, 0, spacing[2]);
>>            extruder.Update();
>>            vtkPolyData extruderOutput = extruder.GetOutput();
>>
>>            vtkPolyDataToImageStencil pol2Stenc = new
>> vtkPolyDataToImageStencil();
>>            pol2Stenc.SetTolerance(0);
>>            pol2Stenc.SetInput(extruderOutput);
>>            pol2Stenc.SetInformationInput(blankImage);
>>            pol2Stenc.Update();
>>            vtkImageStencilData pol2StencOutput = pol2Stenc.GetOutput();
>>
>>            vtkImageStencil stencil = new vtkImageStencil();
>>            stencil.SetInput(blankImage);
>>            stencil.ReverseStencilOn();
>>            stencil.SetStencil(pol2StencOutput);
>>            stencil.Update();
>>            vtkImageData binaryImage = stencil.GetOutput();
>>            binaryImages.add(binaryImage);
>>
>>            long memory = Runtime.getRuntime().totalMemory()/1024/1024;
>>            long shouldMemory =
>> (((extent[1]-extent[0])*(extent[3]-extent[2])*(extent[5]-extent[4])*binaryImages.size())/1024/1024);
>>            System.out.println("Iteration " + i + " using " + memory + " MB");
>>            System.out.println("Should be using: " + shouldMemory + " MB");
>>            System.gc();
>>            System.runFinalization();
>>            vtkGlobalJavaHash.GC();
>>        }
>>    }
>> }
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>
>>
>