[vtk-developers] vtk java memory problem

[Sebastien Jourdain]

Hi Mark,

If you want to make sure no underneath pipeline connection is kept
through your dataset that you keep inside your HashSet, you should use
a deepcopy like in the following lines.

  final vtkImageData binaryImage = stencil.GetOutput();
  final vtkImageData imageToKeep = new vtkImageData();
  imageToKeep.DeepCopy(binaryImage);
  binaryImages.add(imageToKeep);

If you want some internal optimization on the Java GC side, you can
declare all your variables inside the loop "final".

For the VTK GC side, we didn't set it automatically for thread safety
reasons. But you are doing the right thing by calling
"vtkGlobalJavaHash.GC()".
On the other hand System.gc() and System.runFinalization() are not
mandatory and you should pre-allocate your set like that to prevent
reallocation...

   int size = 100;
   HashSet<vtkImageData> binaryImages = new HashSet<vtkImageData>(size);
   for (int i=0; i<size; i++) {
       final vtkImageData blankImage = new vtkImageData();
       ...
    }

Hope this will solved your memory issue,

Seb


On Tue, Dec 28, 2010 at 10:54 PM, Mark Roden <mmroden at gmail.com> wrote:
> Hi all,
>
> I'm trying to track down some memory handling problems when wrapping
> VTK 5.6.1 with Java.  These problems become very severe very quickly
> for what we're trying to do.  I've provided some sample code that
> isolates the issue while still being fundamentally true to the goals
> of the software.  The long and the short of it is that there appear to
> be three different numbers for memory:
> 1) the memory that Java thinks it has
> 2) the memory that we can calculate java should have (not the same as #1)
> 3) the actual memory the process is using (far more than #2).
>
> The application allows for manual delineations of regions of interest
> using contours.  The contours are then extruded and stenciled, and
> somewhere along the line, it looks like memory is leaking.  This is on
> windows 7 x64 using vtk 5.6.1.
>
> To demonstrate the memory problem, this application performs the
> following steps:
> 1) creates a HashSet of vtkImageData
> 2) loops for i to N
> 3) for each loop, create a vtkImageData filled with an extruded and
> stenciled sphere.  Put that image into the HashSet from step 1.
>
> Some of the current pointers for VTK use with Java is to make sure to
> avoid the Set(Get) paradigm, which this code does.
>
> On my machine, the Java runtime thinks it has 183 mb of memory
> allocated, regardless of whether or not the vtkImageData are persisted
> from one iteration to the next (ie, whether or not they are added to
> the HashSet).  The calculated size of the HashSet grows linearly.  The
> actual amount of memory as reported by the Task Manager, however,
> grows exponentially.  Don't run this code unless you can handle 4gb of
> allocation in memory, otherwise killing the process can be painful.
>
> Obviously, if you remove the garbage collection calls, the memory
> amounts will be different, but then the app doesn't get very far at
> all.  We've resigned ourselves to manual garbage collection calls;
> given the size of our allocations, that just appears to be the way of
> the world.  The problem is that even with those manual garbage
> collection calls, memory appears to be leaking still.
>
> Potential culprits here:
> A leak in some portion of the extrusion/stenciling chain
> A leak in garbage collection
> Placing the image into the HashSet is causing multiple copies that
> aren't being tracked properly
> ???
>
> How do I fix this issue?
>
> Thanks,
> Mark
>
>
>
> Here is the code:
>
> import vtk.*;
> import java.util.HashSet;
>
> public class VTKMemoryError {
>
>    static {
>        System.loadLibrary("vtkCommonJava");
>        System.loadLibrary("vtkFilteringJava");
>        System.loadLibrary("vtkGenericFilteringJava");
>        System.loadLibrary("vtkGraphicsJava");
>        System.loadLibrary("vtkHybridJava");
>        System.loadLibrary("vtkImagingJava");
>        System.loadLibrary("vtkIOJava");
>        System.loadLibrary("vtkRenderingJava");
>        System.loadLibrary("vtkVolumeRenderingJava");
>        System.loadLibrary("vtkWidgetsJava");
>    }
>    public static void main(String[] args) {
>        HashSet<vtkImageData> binaryImages = new HashSet<vtkImageData>();
>        for (int i=0; i<100; i++) {
>            vtkImageData blankImage = new vtkImageData();
>            int[] extent = {0, 511, 0, 511, 0, 74};
>            double[] origin = {0, 0, 0};
>            double[] spacing = {1, 1, 3};
>            blankImage.SetExtent(extent);
>            blankImage.SetOrigin(origin);
>            blankImage.SetSpacing(spacing);
>            blankImage.SetScalarTypeToUnsignedChar();
>            blankImage.AllocateScalars();
>            double[] bounds = blankImage.GetBounds();
>
>            vtkSphereSource sphere = new vtkSphereSource();
>            sphere.SetRadius(50);
>            sphere.SetCenter((bounds[0]+bounds[1])/2.0,
> (bounds[2]+bounds[3])/2.0, (bounds[4]+bounds[5])/2.0);
>            sphere.Update();
>            vtkPolyData sphereData = sphere.GetOutput();
>
>            vtkLinearExtrusionFilter extruder = new vtkLinearExtrusionFilter();
>            extruder.SetInput(sphereData);
>            extruder.SetVector(0, 0, spacing[2]);
>            extruder.Update();
>            vtkPolyData extruderOutput = extruder.GetOutput();
>
>            vtkPolyDataToImageStencil pol2Stenc = new
> vtkPolyDataToImageStencil();
>            pol2Stenc.SetTolerance(0);
>            pol2Stenc.SetInput(extruderOutput);
>            pol2Stenc.SetInformationInput(blankImage);
>            pol2Stenc.Update();
>            vtkImageStencilData pol2StencOutput = pol2Stenc.GetOutput();
>
>            vtkImageStencil stencil = new vtkImageStencil();
>            stencil.SetInput(blankImage);
>            stencil.ReverseStencilOn();
>            stencil.SetStencil(pol2StencOutput);
>            stencil.Update();
>            vtkImageData binaryImage = stencil.GetOutput();
>            binaryImages.add(binaryImage);
>
>            long memory = Runtime.getRuntime().totalMemory()/1024/1024;
>            long shouldMemory =
> (((extent[1]-extent[0])*(extent[3]-extent[2])*(extent[5]-extent[4])*binaryImages.size())/1024/1024);
>            System.out.println("Iteration " + i + " using " + memory + " MB");
>            System.out.println("Should be using: " + shouldMemory + " MB");
>            System.gc();
>            System.runFinalization();
>            vtkGlobalJavaHash.GC();
>        }
>    }
> }
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>
>