<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>I think there is an api in the multi-block writer to disable
writing metadata. Utkarsh knows, he's the one that figured it out.
not sure how it applies to your use case. you may not even be
using multi-block format...<br>
</p>
<br>
<div class="moz-cite-prefix">On 05/23/2016 06:57 AM, Gallagher,
Timothy P wrote:<br>
</div>
<blockquote
cite="mid:BN3PR07MB2627F93DC800BA34D03EF05F9D4E0@BN3PR07MB2627.namprd07.prod.outlook.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper"
style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Interesting -- I'd be willing to try it out if you have more
details (commit hashtag, ticket, etc.), even if I have to
figure out how to backport it from a newer version.
<br>
</p>
<p><br>
</p>
<p>Thanks,</p>
<p><br>
</p>
<p>Tim<br>
</p>
<br>
<br>
<div style="color: rgb(49, 55, 57);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
color="#000000" face="Calibri, sans-serif"><b>From:</b>
Burlen Loring <a class="moz-txt-link-rfc2396E" href="mailto:burlen.loring@gmail.com"><burlen.loring@gmail.com></a><br>
<b>Sent:</b> Sunday, May 22, 2016 10:30 PM<br>
<b>To:</b> Gallagher, Timothy P; Andy Bauer<br>
<b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:paraview@paraview.org">paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Memory leak with Catalyst</font>
<div> </div>
</div>
<div>btw, we just had similar problem that was entirely
explained by vtk writer attempting to gather a bunch of
arrays to the root node. Utkarsh found a way to disable that
behavior. I wonder if you are hitting the same.
<br>
<br>
<div class="moz-cite-prefix">On 5/20/2016 4:36 PM,
Gallagher, Timothy P wrote:<br>
</div>
<blockquote type="cite">
<div id="divtagdefaultwrapper" style="font-size:12pt;
color:#000000; background-color:#FFFFFF;
font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Well... not going so well. <br>
</p>
<p><br>
</p>
<p>If I run a small simulation with massif and the
Release build of paraview, I don't see any growing
memory in time. I also get my VTK files and images.
<br>
</p>
<p><br>
</p>
<p>If I run the same simulation linked against a Debug
build with VTK_DEBUG_LEAKS on, there is a segfault
inside one of the libvtkRendering libraries (it says
libvtkRenderingOp but the rest of the name is cut
off). If I take the image rendering out of my pipeline
(comment out the RegisterView call) and leave the VTK
file writing, it runs fine and doesn't report any
leaks. I tried to get the segfault to drop a core
file, but it won't do it. </p>
<p><br>
</p>
<p>I'm thinking that there is an issue in the rendering
part causing the leak. I have no idea what it could be
though -- this is the 4.4.0 version, is anybody aware
of any bug fixes between 4.4.0 and current versions
that might be relevant?
<br>
</p>
<p><br>
</p>
<p>Thanks for the help,</p>
<p><br>
</p>
<p>Tim<br>
</p>
<p><br>
</p>
<p><br>
</p>
<br>
<br>
<div style="color:rgb(0,0,0)">
<hr tabindex="-1" style="display:inline-block;
width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font
style="font-size:11pt" color="#000000"
face="Calibri, sans-serif"><b>From:</b> ParaView
<a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:paraview-bounces@paraview.org"><paraview-bounces@paraview.org></a>
on behalf of Gallagher, Timothy P
<a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:tim.gallagher@gatech.edu"><tim.gallagher@gatech.edu></a><br>
<b>Sent:</b> Friday, May 20, 2016 4:17 PM<br>
<b>To:</b> Burlen Loring; Andy Bauer<br>
<b>Cc:</b> <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:paraview@paraview.org">
paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Memory leak with
Catalyst</font>
<div> </div>
</div>
<div>
<div id="divtagdefaultwrapper"
style="font-size:12pt; color:#000000;
background-color:#FFFFFF;
font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Thanks for the advice Burlen -- I haven't used
Massif before, so this is a good opportunity to
give it a try.
<br>
</p>
<p><br>
</p>
<p>The initialize/finalize each time approach
doesn't work either. I am calling the Finalize()
routine on the vtkCPProcessor class but then
when it tries to initialize again, it throws:</p>
<p><br>
</p>
<div>Warning: In
/data4/ParaView/VTK/Parallel/Core/vtkSocketController.cxx,
line 50<br>
vtkSocketController (0xa4342f0): Already
initialized.<br>
</div>
<p><br>
</p>
<p>and then crashes. So, hopefully I can sort out
the memory leak rather than trying to debug why
a hacked fix doesn't work!</p>
<p><br>
</p>
<p>Tim<br>
</p>
<br>
<br>
<div style="color:rgb(49,55,57)">
<hr tabindex="-1" style="display:inline-block;
width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font
style="font-size:11pt" color="#000000"
face="Calibri, sans-serif"><b>From:</b>
Burlen Loring
<a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:burlen.loring@gmail.com"><burlen.loring@gmail.com></a><br>
<b>Sent:</b> Friday, May 20, 2016 3:46 PM<br>
<b>To:</b> Gallagher, Timothy P; Andy Bauer<br>
<b>Cc:</b> <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:paraview@paraview.org">
paraview@paraview.org</a><br>
<b>Subject:</b> Re: [Paraview] Memory leak
with Catalyst</font>
<div> </div>
</div>
<div>
<p>VTK_DEBUG_LEAKS, although will catch some
serious errors, will not catch all kinds of
leaks. For example you can have leaks where
data erroneously accumulates with each time
step, but is deleted at program termination.
VTK_DEBUG_LEAKS will not catch that kind of
error. It's probably better to use massif to
profile your code on a small one node run.
There's an excellent tool called massif
visualizer to aid in exploring the data
generated.</p>
<p><br>
</p>
<p>$0.02<br>
</p>
<br>
<div class="moz-cite-prefix">On 05/20/2016
11:56 AM, Gallagher, Timothy P wrote:<br>
</div>
<blockquote type="cite">
<div id="divtagdefaultwrapper"
style="font-size:12pt; color:#000000;
background-color:#FFFFFF;
font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi Andy,</p>
<p><br>
</p>
<p>Thanks for the tips. I will get working
on the VTK_DEBUG_LEAKS now and see what
it turns up.
<br>
</p>
<p><br>
</p>
<p>The initialize/finalize every time is
definitely a hack and not for long-term
use. But, sponsors want a report on
Monday and we need to be able to
visualize things for that -- the
simulation is too big to write data
files and post-process later. So I
modified the code to do that for now
until I can find a proper fix. <br>
</p>
<p><br>
</p>
<p>I'll let you know if I get stuck with
the log file. <br>
</p>
<p><br>
</p>
<p>Thanks again,</p>
<p><br>
</p>
<p>Tim<br>
</p>
<br>
<br>
<div style="color:rgb(49,55,57)">
<hr tabindex="-1"
style="display:inline-block;
width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font
style="font-size:11pt"
color="#000000" face="Calibri,
sans-serif"><b>From:</b> Andy Bauer
<a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:andy.bauer@kitware.com"><andy.bauer@kitware.com></a><br>
<b>Sent:</b> Friday, May 20, 2016
2:39 PM<br>
<b>To:</b> Gallagher, Timothy P<br>
<b>Cc:</b> <a
moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:paraview@paraview.org">
<a class="moz-txt-link-abbreviated" href="mailto:paraview@paraview.org">paraview@paraview.org</a></a><br>
<b>Subject:</b> Re: [Paraview]
Memory leak with Catalyst</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>Hi Tim,<br>
<br>
</div>
If you build Catalyst with
VTK_DEBUG_LEAKS enabled it is
pretty good at finding VTK
objects that aren't deleted
properly. You should be able
to run this with a small
amount of calls to Catalyst as
well. If you try this and want
help understanding the output
(if an object like a
vtkUnstructuredGrid is leaked
it will often give a whole
bunch of false leaks that the
unstructured grid is just
holding the references to),
just share your output file
and I can take a look at it to
try and narrow down the
culprit. That may be slightly
easier to use than Valgrind.<br>
<br>
</div>
<div>Beyond this, you could
maybe try the same run but
without doing any Catalyst
work to see what happens then.
That may be a lot of compute
cycles but I'm not sure how
else to try and bisect where
the memory leak is occurring.<br>
</div>
<div><br>
</div>
Initializing and finalizing
Catalyst every time you want
output would probably work but
I'd think there may be
significant overhead doing it
like this. Plus, it's not really
solving the problem -- just
avoiding it.<br>
<br>
</div>
Best,<br>
</div>
Andy<br>
<div>
<div>
<div><br>
</div>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, May
20, 2016 at 12:57 PM, Gallagher,
Timothy P <span dir="ltr">
<<a moz-do-not-send="true"
href="mailto:tim.gallagher@gatech.edu"
target="_blank">tim.gallagher@gatech.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;
border-left:1px #ccc solid;
padding-left:1ex">
<div dir="ltr">
<div style="font-size:12pt;
color:#000000;
background-color:#ffffff;
font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi,</p>
<p><br>
</p>
<p>One of our users is
running a very big
simulation and writing out
images of two slices (two
different views) every
1000 iterations and
writing out the data for
the two slices (two
different data writers) as
VTK files every 5000
iterations. It is using
Paraview 4.4. <br>
</p>
<p><br>
</p>
<p>After 21000 iterations,
the simulation is killed
because the memory on the
compute nodes fills up. I
usually know how to track
down memory problems in
our code using valgrind
and related tools, but is
that the right way to go
to try and find this
problem?</p>
<p><br>
</p>
<p>Are there any tips on how
to isolate where the
problem may be? I don't
know if it is in the
adapter, or in paraview
itself. Has anybody
encountered problems with
runaway memory using
Catalyst in 4.4 when
writing images or VTK
files?
<br>
</p>
<p><br>
</p>
<p>I know when we use
pvpython to generate
images and loop over many
files, sometimes the
memory also blows up and
so we usually move the
loop over the files
outside the pvpython
script and into a driver
script that executes a new
pvpython for each file. Is
there a way to shut
down/start up Catalyst
each time it needs to
write something? Is that
advisable?
<br>
</p>
<p><br>
</p>
<p>Thanks,</p>
<p><br>
</p>
<p>Tim<br>
</p>
</div>
</div>
<br>
_______________________________________________<br>
Powered by <a
moz-do-not-send="true"
href="http://www.kitware.com"
rel="noreferrer"
target="_blank"><a class="moz-txt-link-abbreviated" href="http://www.kitware.com">www.kitware.com</a></a><br>
<br>
Visit other Kitware open-source
projects at <a
moz-do-not-send="true"
href="http://www.kitware.com/opensource/opensource.html"
rel="noreferrer"
target="_blank">
</a><a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic
and check the ParaView Wiki at:
<a moz-do-not-send="true"
href="http://paraview.org/Wiki/ParaView"
rel="noreferrer"
target="_blank">
</a><a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><br>
<br>
Search the list archives at: <a
moz-do-not-send="true"
href="http://markmail.org/search/?q=ParaView"
rel="noreferrer"
target="_blank">
</a><a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://markmail.org/search/?q=ParaView">http://markmail.org/search/?q=ParaView</a><br>
<br>
Follow this link to
subscribe/unsubscribe:<br>
<a moz-do-not-send="true"
href="http://public.kitware.com/mailman/listinfo/paraview"
rel="noreferrer"
target="_blank">http://public.kitware.com/mailman/listinfo/paraview</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Powered by <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="http://www.kitware.com">www.kitware.com</a>
Visit other Kitware open-source projects at <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a>
Please keep messages on-topic and check the ParaView Wiki at: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a>
Search the list archives at: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://markmail.org/search/?q=ParaView">http://markmail.org/search/?q=ParaView</a>
Follow this link to subscribe/unsubscribe:
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://public.kitware.com/mailman/listinfo/paraview">http://public.kitware.com/mailman/listinfo/paraview</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>