[vtk-developers] vtk java memory problem

[Mark Roden]

Hi all,

I'm trying to track down some memory handling problems when wrapping
VTK 5.6.1 with Java.  These problems become very severe very quickly
for what we're trying to do.  I've provided some sample code that
isolates the issue while still being fundamentally true to the goals
of the software.  The long and the short of it is that there appear to
be three different numbers for memory:
1) the memory that Java thinks it has
2) the memory that we can calculate java should have (not the same as #1)
3) the actual memory the process is using (far more than #2).

The application allows for manual delineations of regions of interest
using contours.  The contours are then extruded and stenciled, and
somewhere along the line, it looks like memory is leaking.  This is on
windows 7 x64 using vtk 5.6.1.

To demonstrate the memory problem, this application performs the
following steps:
1) creates a HashSet of vtkImageData
2) loops for i to N
3) for each loop, create a vtkImageData filled with an extruded and
stenciled sphere.  Put that image into the HashSet from step 1.

Some of the current pointers for VTK use with Java is to make sure to
avoid the Set(Get) paradigm, which this code does.

On my machine, the Java runtime thinks it has 183 mb of memory
allocated, regardless of whether or not the vtkImageData are persisted
from one iteration to the next (ie, whether or not they are added to
the HashSet).  The calculated size of the HashSet grows linearly.  The
actual amount of memory as reported by the Task Manager, however,
grows exponentially.  Don't run this code unless you can handle 4gb of
allocation in memory, otherwise killing the process can be painful.

Obviously, if you remove the garbage collection calls, the memory
amounts will be different, but then the app doesn't get very far at
all.  We've resigned ourselves to manual garbage collection calls;
given the size of our allocations, that just appears to be the way of
the world.  The problem is that even with those manual garbage
collection calls, memory appears to be leaking still.

Potential culprits here:
A leak in some portion of the extrusion/stenciling chain
A leak in garbage collection
Placing the image into the HashSet is causing multiple copies that
aren't being tracked properly
???

How do I fix this issue?

Thanks,
Mark



Here is the code:

import vtk.*;
import java.util.HashSet;

public class VTKMemoryError {

    static {
        System.loadLibrary("vtkCommonJava");
        System.loadLibrary("vtkFilteringJava");
        System.loadLibrary("vtkGenericFilteringJava");
        System.loadLibrary("vtkGraphicsJava");
        System.loadLibrary("vtkHybridJava");
        System.loadLibrary("vtkImagingJava");
        System.loadLibrary("vtkIOJava");
        System.loadLibrary("vtkRenderingJava");
        System.loadLibrary("vtkVolumeRenderingJava");
        System.loadLibrary("vtkWidgetsJava");
    }
    public static void main(String[] args) {
        HashSet<vtkImageData> binaryImages = new HashSet<vtkImageData>();
        for (int i=0; i<100; i++) {
            vtkImageData blankImage = new vtkImageData();
            int[] extent = {0, 511, 0, 511, 0, 74};
            double[] origin = {0, 0, 0};
            double[] spacing = {1, 1, 3};
            blankImage.SetExtent(extent);
            blankImage.SetOrigin(origin);
            blankImage.SetSpacing(spacing);
            blankImage.SetScalarTypeToUnsignedChar();
            blankImage.AllocateScalars();
            double[] bounds = blankImage.GetBounds();

            vtkSphereSource sphere = new vtkSphereSource();
            sphere.SetRadius(50);
            sphere.SetCenter((bounds[0]+bounds[1])/2.0,
(bounds[2]+bounds[3])/2.0, (bounds[4]+bounds[5])/2.0);
            sphere.Update();
            vtkPolyData sphereData = sphere.GetOutput();

            vtkLinearExtrusionFilter extruder = new vtkLinearExtrusionFilter();
            extruder.SetInput(sphereData);
            extruder.SetVector(0, 0, spacing[2]);
            extruder.Update();
            vtkPolyData extruderOutput = extruder.GetOutput();

            vtkPolyDataToImageStencil pol2Stenc = new
vtkPolyDataToImageStencil();
            pol2Stenc.SetTolerance(0);
            pol2Stenc.SetInput(extruderOutput);
            pol2Stenc.SetInformationInput(blankImage);
            pol2Stenc.Update();
            vtkImageStencilData pol2StencOutput = pol2Stenc.GetOutput();

            vtkImageStencil stencil = new vtkImageStencil();
            stencil.SetInput(blankImage);
            stencil.ReverseStencilOn();
            stencil.SetStencil(pol2StencOutput);
            stencil.Update();
            vtkImageData binaryImage = stencil.GetOutput();
            binaryImages.add(binaryImage);

            long memory = Runtime.getRuntime().totalMemory()/1024/1024;
            long shouldMemory =
(((extent[1]-extent[0])*(extent[3]-extent[2])*(extent[5]-extent[4])*binaryImages.size())/1024/1024);
            System.out.println("Iteration " + i + " using " + memory + " MB");
            System.out.println("Should be using: " + shouldMemory + " MB");
            System.gc();
            System.runFinalization();
            vtkGlobalJavaHash.GC();
        }
    }
}