[vtkusers] Memory not released when volumes are removed

[Elvis Stansvik]

2016-09-01 13:57 GMT+02:00 Elvis Stansvik <elvis.stansvik at orexplore.com>:

> 2016-09-01 9:43 GMT+02:00 Elvis Stansvik <elvis.stansvik at orexplore.com>:
>
>> 2016-09-01 9:31 GMT+02:00 Róbert Špir <spir.robert at gmail.com>:
>>
>>> try building vtk with VTK_DEBUG_LEAKS enabled and then you can use
>>> vtkDebugLeaks class PrintCurrentLeaks method to print all existing vtk
>>> objects. Here you can see whether the object count is increasing or
>>> decreasing
>>>
>>
>> Ah yes, I'll do that. In the meantime I'm trying to debug with Valgrind
>> Massif, and I have found some strangeness (have to debug further, but
>> there's definitely real leaks going on). Thanks for the tips.
>>
>> Elvis
>>
>>
>>>
>>>
>>> *From:* Elvis Stansvik [mailto:elvis.stansvik at orexplore.com]
>>> *Sent:* Thursday, September 1, 2016 9:05 AM
>>> *To:* RobertS <spir.robert at gmail.com>
>>> *Cc:* VTK Users <vtkusers at vtk.org>
>>> *Subject:* Re: [vtkusers] Memory not released when volumes are removed
>>>
>>>
>>>
>>> 2016-08-31 17:51 GMT+02:00 Elvis Stansvik <elvis.stansvik at orexplore.com>
>>> :
>>>
>>> HI Robert,
>>>
>>> Den 31 aug. 2016 5:30 em skrev "Róbert Špir" <spir.robert at gmail.com>:
>>> >
>>> > Hi Elvis,
>>> >
>>> > I noticed the same behavior before ( here is the topic
>>> http://vtk.1045678.n5.nabble.com/memory-leak-in-vtkStructure
>>> dPoints-or-am-I-doing-something-wrong-tt5738805.html )
>>> >
>>> > Looks like it is the standard behavior of Linux (although very weird),
>>> since this is not happening in Windows or OS X.
>>>
>>> Ah right, now I remember your post. I'll do some more testing to verify
>>> that this is actually what's happening in my case, but it likely is.
>>>
>>> I don't think this is the phenomenon I'm seeing. I tried opening the
>>> same dataset repeatedly in my app, and I got:
>>>
>>> 77 MB
>>> 368 MB
>>> 633 MB
>>> 860 MB
>>> 1003 MB
>>> 1171 MB
>>> 1345 MB
>>> 1518 MB
>>> 1691 MB
>>> 1858 MB
>>> 2029 MB
>>> 2201 MB
>>> 2371 MB
>>> 2543 MB
>>>
>>> So the memory usage just keeps growing, despite my attempts to clean up
>>> before loading the dataset. This is from the RES (resident) column of top,
>>> and I don't think this is memory which can be reclaimed by the OS? That
>>> would show up in the VIRT (virtual) column I think.
>>>
>>> So my question remains: Should I do something more to get a thorough
>>> cleanup, apart from removing the vtkVolume from the renderer and deleting
>>> it (by means of a smart pointer)?
>>>
>> I found the problem, and it was two silly mistakes by me.
>
> The first was plainly visible in the snippet in my first post: I can't use
> a call to GetNumberOfItems() in the loop condition, since when the
> vtkVolume is destroyed, the vtkVolumeCollection will observe this and
> remove the volume from the collection, thus reducing the number of items by
> one in each iteration. So with my test dataset (3 volumes), only two were
> destroyed. Saving the result of GetNumberOfItems() before starting the
> iteration solved that problem.
>
> The second mistake was that I was making use of QtConcurrent::mapped to
> load the volumes in parallel on the Qt thread pool, and then setting up a
> QFutureWatcher on the resulting QFuture, with a lambda connected to the
> finished() signal of the QFutureWatcher. In the lambda I set up the
> visualization parts of the pipeline and added the volumes to the renderer.
> I was reusing a single QFutureWatcher for this, which would have been fine
> if it wasn't for the fact that I connect()ed the lambda to its finished()
> signal each time I loaded a new dataset. So each time the number of
> connections increased by one, causing the lambda one extra time each time a
> new dataset was loaded.
>

*causing the lambda to be called [...]

Elvis


> The solution I opted for was to instead create a new QFutureWatcher each
> time a data set is loaded, and then also make sure the QFutureWatcher is
> destroyed when it finishes by connecting finished() to deleteLater().
>
> The memory use is no longer growing when I repeatedly load new datasets.
>
> Elvis
>
>
>> Elvis
>>>
>>>
>>>
>>> Thanks,
>>> Elvis
>>>
>>> >
>>> > Best,
>>> >
>>> > Robert
>>> >
>>> >
>>> >
>>> > From: vtkusers [mailto:vtkusers-bounces at vtk.org] On Behalf Of Elvis
>>> Stansvik
>>> > Sent: Wednesday, August 31, 2016 5:19 PM
>>> > To: VTK Users <vtkusers at vtk.org>
>>> > Subject: [vtkusers] Memory not released when volumes are removed
>>> >
>>> >
>>> >
>>> > Hi all,
>>> >
>>> > Before loading a new dataset into our app, I'd like to remove all
>>> existing volumes from the renderer, which I believe should lead to them
>>> being destroyed as well, since I have no other references to the volumes
>>> (currently).
>>> >
>>> > What I tried was
>>> >
>>> >     // Remove all existing volumes
>>> >     auto volumes = renderer()->GetVolumes();
>>> >     volumes->InitTraversal();
>>> >     for (int i = 0; i < volumes->GetNumberOfItems(); ++i) {
>>> >         qDebug() << "removing volume";
>>> >         vtkSmartPointer<vtkVolume> volume = volumes->GetNextVolume();
>>> >         qDebug() << "before:" << volume->GetReferenceCount();
>>> >         renderer()->RemoveVolume(volume.GetPointer());
>>> >         qDebug() << "after:" << volume->GetReferenceCount();
>>> >     }
>>> >
>>> > The output I get each time I load a new dataset (actually the same,
>>> but doing it repeatedly) and execution passes by this cleanup code is:
>>> >
>>> > removing volume
>>> > before: 3
>>> > after: 1
>>> > removing volume
>>> > before: 3
>>> > after: 1
>>> > removing volume
>>> > before: 3
>>> > after: 1
>>> >
>>> > Which sort of makes sense to me: Apparently the renderer holds two
>>> references to the volume, since the refcount goes down by 2 when I remove
>>> the volume from the renderer. The remaining reference is held by the
>>> vtkSmartPointer I created inside the loop.
>>> >
>>> > But, what surprises me is that memory usage does not seem to go down
>>> when this code is executed. I would expect that when the scope of the for
>>> loop body is exited, the refcount will drop to zero, so the volume should
>>> be deleted. And I guess it probably is.
>>> >
>>> > But looking at the memory usage when repeatedly loading datasets (and
>>> thus clearing the previous), the resident memory used by the app
>>> accumulates.
>>> >
>>> > Must I somehow remove the mapper / underlying image data as well? I'm
>>> not holding any references to these, so I would have expected them to die
>>> along with the volume.
>>> >
>>> > Helpful for any tips on why this cleanup is apparently not enough to
>>> avoid memory accumulation.
>>> >
>>> > Thanks,
>>> >
>>> > Elvis
>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20160901/4fb75f32/attachment.html>