[Paraview] Memory leak with Catalyst

Sun May 22 22:30:43 EDT 2016

btw, we just had similar problem that was entirely explained by vtk 
writer attempting to gather a bunch of arrays to the root node. Utkarsh 
found a way to disable that behavior. I wonder if you are hitting the same.

On 5/20/2016 4:36 PM, Gallagher, Timothy P wrote:
>
> Well... not going so well.
>
>
> If I run a small simulation with massif and the Release build of 
> paraview, I don't see any growing memory in time. I also get my VTK 
> files and images.
>
>
> If I run the same simulation linked against a Debug build with 
> VTK_DEBUG_LEAKS on, there is a segfault inside one of the 
> libvtkRendering libraries (it says libvtkRenderingOp but the rest of 
> the name is cut off). If I take the image rendering out of my pipeline 
> (comment out the RegisterView call) and leave the VTK file writing, it 
> runs fine and doesn't report any leaks. I tried to get the segfault to 
> drop a core file, but it won't do it.
>
>
> I'm thinking that there is an issue in the rendering part causing the 
> leak. I have no idea what it could be though -- this is the 4.4.0 
> version, is anybody aware of any bug fixes between 4.4.0 and current 
> versions that might be relevant?
>
>
> Thanks for the help,
>
>
> Tim
>
>
>
>
>
> ------------------------------------------------------------------------
> *From:* ParaView <paraview-bounces at paraview.org> on behalf of 
> Gallagher, Timothy P <tim.gallagher at gatech.edu>
> *Sent:* Friday, May 20, 2016 4:17 PM
> *To:* Burlen Loring; Andy Bauer
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Memory leak with Catalyst
>
> Thanks for the advice Burlen -- I haven't used Massif before, so this 
> is a good opportunity to give it a try.
>
>
> The initialize/finalize each time approach doesn't work either. I am 
> calling the Finalize() routine on the vtkCPProcessor class but then 
> when it tries to initialize again, it throws:
>
>
> Warning: In /data4/ParaView/VTK/Parallel/Core/vtkSocketController.cxx, 
> line 50
> vtkSocketController (0xa4342f0): Already initialized.
>
>
> and then crashes. So, hopefully I can sort out the memory leak rather 
> than trying to debug why a hacked fix doesn't work!
>
>
> Tim
>
>
>
> ------------------------------------------------------------------------
> *From:* Burlen Loring <burlen.loring at gmail.com>
> *Sent:* Friday, May 20, 2016 3:46 PM
> *To:* Gallagher, Timothy P; Andy Bauer
> *Cc:* paraview at paraview.org
> *Subject:* Re: [Paraview] Memory leak with Catalyst
>
> VTK_DEBUG_LEAKS, although will catch some serious errors, will not 
> catch all kinds of leaks. For example you can have leaks where data 
> erroneously accumulates with each time step, but is deleted at program 
> termination. VTK_DEBUG_LEAKS will not catch that kind of error. It's 
> probably better to use massif to profile your code on a small one node 
> run. There's an excellent tool called massif visualizer to aid in 
> exploring the data generated.
>
>
> $0.02
>
>
> On 05/20/2016 11:56 AM, Gallagher, Timothy P wrote:
>>
>> Hi Andy,
>>
>>
>> Thanks for the tips. I will get working on the VTK_DEBUG_LEAKS now 
>> and see what it turns up.
>>
>>
>> The initialize/finalize every time is definitely a hack and not for 
>> long-term use. But, sponsors want a report on Monday and we need to 
>> be able to visualize things for that -- the simulation is too big to 
>> write data files and post-process later. So I modified the code to do 
>> that for now until I can find a proper fix.
>>
>>
>> I'll let you know if I get stuck with the log file.
>>
>>
>> Thanks again,
>>
>>
>> Tim
>>
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Andy Bauer <andy.bauer at kitware.com>
>> *Sent:* Friday, May 20, 2016 2:39 PM
>> *To:* Gallagher, Timothy P
>> *Cc:* paraview at paraview.org
>> *Subject:* Re: [Paraview] Memory leak with Catalyst
>> Hi Tim,
>>
>> If you build Catalyst with VTK_DEBUG_LEAKS enabled it is pretty good 
>> at finding VTK objects that aren't deleted properly. You should be 
>> able to run this with a small amount of calls to Catalyst as well. If 
>> you try this and want help understanding the output (if an object 
>> like a vtkUnstructuredGrid is leaked it will often give a whole bunch 
>> of false leaks that the unstructured grid is just holding the 
>> references to), just share your output file and I can take a look at 
>> it to try and narrow down the culprit. That may be slightly easier to 
>> use than Valgrind.
>>
>> Beyond this, you could maybe try the same run but without doing any 
>> Catalyst work to see what happens then. That may be a lot of compute 
>> cycles but I'm not sure how else to try and bisect where the memory 
>> leak is occurring.
>>
>> Initializing and finalizing Catalyst every time you want output would 
>> probably work but I'd think there may be significant overhead doing 
>> it like this. Plus, it's not really solving the problem -- just 
>> avoiding it.
>>
>> Best,
>> Andy
>>
>>
>> On Fri, May 20, 2016 at 12:57 PM, Gallagher, Timothy P 
>> <tim.gallagher at gatech.edu <mailto:tim.gallagher at gatech.edu>> wrote:
>>
>>     Hi,
>>
>>
>>     One of our users is running a very big simulation and writing out
>>     images of two slices (two different views) every 1000 iterations
>>     and writing out the data for the two slices (two different data
>>     writers) as VTK files every 5000 iterations. It is using Paraview
>>     4.4.
>>
>>
>>     After 21000 iterations, the simulation is killed because the
>>     memory on the compute nodes fills up. I usually know how to track
>>     down memory problems in our code using valgrind and related
>>     tools, but is that the right way to go to try and find this problem?
>>
>>
>>     Are there any tips on how to isolate where the problem may be? I
>>     don't know if it is in the adapter, or in paraview itself. Has
>>     anybody encountered problems with runaway memory using Catalyst
>>     in 4.4 when writing images or VTK files?
>>
>>
>>     I know when we use pvpython to generate images and loop over many
>>     files, sometimes the memory also blows up and so we usually move
>>     the loop over the files outside the pvpython script and into a
>>     driver script that executes a new pvpython for each file. Is
>>     there a way to shut down/start up Catalyst each time it needs to
>>     write something? Is that advisable?
>>
>>
>>     Thanks,
>>
>>
>>     Tim
>>
>>
>>     _______________________________________________
>>     Powered by www.kitware.com <http://www.kitware.com>
>>
>>     Visit other Kitware open-source projects at
>>     <http://www.kitware.com/opensource/opensource.html>http://www.kitware.com/opensource/opensource.html
>>
>>     Please keep messages on-topic and check the ParaView Wiki at:
>>     http://paraview.org/Wiki/ParaView <http://paraview.org/Wiki/ParaView>
>>
>>     Search the list archives at:
>>     <http://markmail.org/search/?q=ParaView>http://markmail.org/search/?q=ParaView
>>
>>     Follow this link to subscribe/unsubscribe:
>>     http://public.kitware.com/mailman/listinfo/paraview
>>
>>
>>
>>
>> _______________________________________________
>> Powered bywww.kitware.com
>>
>> Visit other Kitware open-source projects athttp://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at:http://paraview.org/Wiki/ParaView
>>
>> Search the list archives at:http://markmail.org/search/?q=ParaView
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/paraview
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20160522/8b578a42/attachment.html>