[vtkusers] Memory not released when volumes are removed

Thu Sep 1 07:57:01 EDT 2016

2016-09-01 9:43 GMT+02:00 Elvis Stansvik <elvis.stansvik at orexplore.com>:

> 2016-09-01 9:31 GMT+02:00 Róbert Špir <spir.robert at gmail.com>:
>
>> try building vtk with VTK_DEBUG_LEAKS enabled and then you can use
>> vtkDebugLeaks class PrintCurrentLeaks method to print all existing vtk
>> objects. Here you can see whether the object count is increasing or
>> decreasing
>>
>
> Ah yes, I'll do that. In the meantime I'm trying to debug with Valgrind
> Massif, and I have found some strangeness (have to debug further, but
> there's definitely real leaks going on). Thanks for the tips.
>
> Elvis
>
>
>>
>>
>> *From:* Elvis Stansvik [mailto:elvis.stansvik at orexplore.com]
>> *Sent:* Thursday, September 1, 2016 9:05 AM
>> *To:* RobertS <spir.robert at gmail.com>
>> *Cc:* VTK Users <vtkusers at vtk.org>
>> *Subject:* Re: [vtkusers] Memory not released when volumes are removed
>>
>>
>>
>> 2016-08-31 17:51 GMT+02:00 Elvis Stansvik <elvis.stansvik at orexplore.com>:
>>
>> HI Robert,
>>
>> Den 31 aug. 2016 5:30 em skrev "Róbert Špir" <spir.robert at gmail.com>:
>> >
>> > Hi Elvis,
>> >
>> > I noticed the same behavior before ( here is the topic
>> http://vtk.1045678.n5.nabble.com/memory-leak-in-vtkStructure
>> dPoints-or-am-I-doing-something-wrong-tt5738805.html )
>> >
>> > Looks like it is the standard behavior of Linux (although very weird),
>> since this is not happening in Windows or OS X.
>>
>> Ah right, now I remember your post. I'll do some more testing to verify
>> that this is actually what's happening in my case, but it likely is.
>>
>> I don't think this is the phenomenon I'm seeing. I tried opening the same
>> dataset repeatedly in my app, and I got:
>>
>> 77 MB
>> 368 MB
>> 633 MB
>> 860 MB
>> 1003 MB
>> 1171 MB
>> 1345 MB
>> 1518 MB
>> 1691 MB
>> 1858 MB
>> 2029 MB
>> 2201 MB
>> 2371 MB
>> 2543 MB
>>
>> So the memory usage just keeps growing, despite my attempts to clean up
>> before loading the dataset. This is from the RES (resident) column of top,
>> and I don't think this is memory which can be reclaimed by the OS? That
>> would show up in the VIRT (virtual) column I think.
>>
>> So my question remains: Should I do something more to get a thorough
>> cleanup, apart from removing the vtkVolume from the renderer and deleting
>> it (by means of a smart pointer)?
>>
> I found the problem, and it was two silly mistakes by me.

The first was plainly visible in the snippet in my first post: I can't use
a call to GetNumberOfItems() in the loop condition, since when the
vtkVolume is destroyed, the vtkVolumeCollection will observe this and
remove the volume from the collection, thus reducing the number of items by
one in each iteration. So with my test dataset (3 volumes), only two were
destroyed. Saving the result of GetNumberOfItems() before starting the
iteration solved that problem.

The second mistake was that I was making use of QtConcurrent::mapped to
load the volumes in parallel on the Qt thread pool, and then setting up a
QFutureWatcher on the resulting QFuture, with a lambda connected to the
finished() signal of the QFutureWatcher. In the lambda I set up the
visualization parts of the pipeline and added the volumes to the renderer.
I was reusing a single QFutureWatcher for this, which would have been fine
if it wasn't for the fact that I connect()ed the lambda to its finished()
signal each time I loaded a new dataset. So each time the number of
connections increased by one, causing the lambda one extra time each time a
new dataset was loaded. The solution I opted for was to instead create a
new QFutureWatcher each time a data set is loaded, and then also make sure
the QFutureWatcher is destroyed when it finishes by connecting finished()
to deleteLater().

The memory use is no longer growing when I repeatedly load new datasets.

Elvis

> Elvis
>>
>>
>>
>> Thanks,
>> Elvis
>>
>> >
>> > Best,
>> >
>> > Robert
>> >
>> >
>> >
>> > From: vtkusers [mailto:vtkusers-bounces at vtk.org] On Behalf Of Elvis
>> Stansvik
>> > Sent: Wednesday, August 31, 2016 5:19 PM
>> > To: VTK Users <vtkusers at vtk.org>
>> > Subject: [vtkusers] Memory not released when volumes are removed
>> >
>> >
>> >
>> > Hi all,
>> >
>> > Before loading a new dataset into our app, I'd like to remove all
>> existing volumes from the renderer, which I believe should lead to them
>> being destroyed as well, since I have no other references to the volumes
>> (currently).
>> >
>> > What I tried was
>> >
>> >     // Remove all existing volumes
>> >     auto volumes = renderer()->GetVolumes();
>> >     volumes->InitTraversal();
>> >     for (int i = 0; i < volumes->GetNumberOfItems(); ++i) {
>> >         qDebug() << "removing volume";
>> >         vtkSmartPointer<vtkVolume> volume = volumes->GetNextVolume();
>> >         qDebug() << "before:" << volume->GetReferenceCount();
>> >         renderer()->RemoveVolume(volume.GetPointer());
>> >         qDebug() << "after:" << volume->GetReferenceCount();
>> >     }
>> >
>> > The output I get each time I load a new dataset (actually the same, but
>> doing it repeatedly) and execution passes by this cleanup code is:
>> >
>> > removing volume
>> > before: 3
>> > after: 1
>> > removing volume
>> > before: 3
>> > after: 1
>> > removing volume
>> > before: 3
>> > after: 1
>> >
>> > Which sort of makes sense to me: Apparently the renderer holds two
>> references to the volume, since the refcount goes down by 2 when I remove
>> the volume from the renderer. The remaining reference is held by the
>> vtkSmartPointer I created inside the loop.
>> >
>> > But, what surprises me is that memory usage does not seem to go down
>> when this code is executed. I would expect that when the scope of the for
>> loop body is exited, the refcount will drop to zero, so the volume should
>> be deleted. And I guess it probably is.
>> >
>> > But looking at the memory usage when repeatedly loading datasets (and
>> thus clearing the previous), the resident memory used by the app
>> accumulates.
>> >
>> > Must I somehow remove the mapper / underlying image data as well? I'm
>> not holding any references to these, so I would have expected them to die
>> along with the volume.
>> >
>> > Helpful for any tips on why this cleanup is apparently not enough to
>> avoid memory accumulation.
>> >
>> > Thanks,
>> >
>> > Elvis
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20160901/e23b7444/attachment.html>