<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Nikolaos,<br>
<br>
One thing that I would suggest is to restructure your script so that
you do not create new objects inside your loop. You can usually
construct your rendering pipeline first then loop over the
files(times, arrays, etc) modifying the existing pipeline objects
and then calling Render followed by WriteImage. When you write your
script that way you avoid the possibility of creating leaks. If you
are convinced that there is a leak in ParaView we could advise on
locating it, however I hesitate to give that advise prematurely.
It's more likely that the issues are resulting from the way your
script is structured.<br>
<br>
Burlen<br>
<br>
<div class="moz-cite-prefix">On 01/08/2015 09:57 AM, Nikolaos
Beratlis wrote:<br>
</div>
<blockquote
cite="mid:CALdMxnyj7Wf2e2YBoZ_X1Dm1ugwe-V_RZ+J3vGU6FexE5FjP_g@mail.gmail.com"
type="cite">
<div dir="ltr">After reading about memory leaks I modified the
script at the end by adding "del" after "Delete":
<div><br>
</div>
<div>
<div> Delete(DataRepresentation4)</div>
<div> del DataRepresentation4</div>
<div> Delete(Contour2)</div>
<div> del Contour2</div>
<div> Delete(DataRepresentation3)</div>
<div> del DataRepresentation3</div>
<div> Delete(Q_VP_xmf)</div>
<div> del Q_VP_xmf</div>
<div> Delete(DataRepresentation2)</div>
<div> del DataRepresentation2</div>
<div> Delete(Contour1)</div>
<div> del Contour1</div>
<div> Delete(DataRepresentation1)</div>
<div> del DataRepresentation1</div>
<div> Delete(cf_VP_xmf)</div>
<div> del cf_VP_xmf</div>
</div>
<div><br>
</div>
<div>That should fully delete the objects from the memory and it
helped a little but I still have memory leaks. Before the
script memory usage in the Memory Inspector is 1.4Gb and after
one iteration it increases to 4Gb, compared to 10Gb without
the del statements. After 4 iterations the memory usage
increases to 8Gb. Any ideas on how to track and eliminate
completely the memory leaks? I am forgetting to delete an
object from the script?</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Jan 7, 2015 at 9:17 PM,
Nikolaos Beratlis <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:nikos.beratlis@gmail.com" target="_blank">nikos.beratlis@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">So I did some more tests by running the
script interactively. I used a smaller grid first, 2002 x
20 x 1000 points and ran it only for 1 iteration. When
Paraview loaded the memory usage on the server was 1.04GB.
After doing one iteration the memory usage spiked to
10.42GB and remained like that when the script finished.
What I found out is the Delete statements make no
difference at all, with or without them the memory usage
remained 10.42GB. Performing more iterations in the script
actually worsens this problem, after 4 iterations the
memory used increased to 14.32GB. With a larger grid of
2002 x 100 x 1000 points the memory after one iteration
increased from 10.42GB before the script to 45.25GB after
the script. How can I properly delete all datasets and
clear the memory in Paraview?
<div><br>
</div>
<div><br>
</div>
</div>
<div class="HOEnZb">
<div class="h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Jan 7, 2015 at 2:55
PM, Nikolaos Beratlis <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:nikos.beratlis@gmail.com"
target="_blank">nikos.beratlis@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I am trying to run a script on a
cluster with Paraview in parallel. The cluster
architecture is 16cores and 64GB per node. The
script reads in two XMF files, one contains a
grid of 2002x100x1000 points along with one
variable. There is a loop to perform two
iterations:
<div><br>
</div>
<div>
<div>try: paraview.simple</div>
<div>except: from paraview.simple import *</div>
<div>paraview.simple._DisableFirstRenderCameraReset()</div>
<div><br>
</div>
<div>RenderView1 = GetRenderView()</div>
<div>RenderView1.Background = [1.0, 1.0, 1.0]</div>
<div>view = GetActiveView()</div>
<div>view.ViewSize = [ 800, 500 ]</div>
<div><br>
</div>
<div>RenderView1.CenterAxesVisibility = 0</div>
<div>#RenderView1.OrientationAxesVisibility =
0</div>
<div><br>
</div>
<div>#Define variables</div>
<div>wdir =
'/lustre/groups/balarasgrp/nikosb/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm'</div>
<div>#wdir =
'/Users/nikosb/colone/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm'</div>
<div>#wdir =
'/Users/nikosb/Research/Simulations/SPHERE/Re100/gridC1/run'</div>
<div>Qval = 0.1</div>
<div>cfmin = -0.001</div>
<div>cfmax = 0.005</div>
<div><br>
</div>
<div>index = [None]*9999</div>
<div><br>
</div>
<div>m = 0</div>
<div>for i in range(0, 9):</div>
<div> for j in range(0, 9):</div>
<div> for k in range(0, 9):</div>
<div> for l in range(0, 9):</div>
<div> index[m] =
chr(i+48)+chr(j+48)+chr(k+48)+chr(l+48)</div>
<div> m = m+1</div>
<div><br>
</div>
<div><br>
</div>
<div>for i in range(1,3):</div>
<div><br>
</div>
<div> cf_VP_xmf = XDMFReader(
FileName=wdir+'/VPfield/XDMF/cf_VP'+index[i]+'.xmf'
)</div>
<div><br>
</div>
<div> cf_VP_xmf.Sets = []</div>
<div> cf_VP_xmf.Grids = ['STL Grid']</div>
<div> cf_VP_xmf.PointArrays = ['Cf']</div>
<div><br>
</div>
<div> RenderView1 = GetRenderView()</div>
<div> RenderView1.CenterOfRotation =
[6.2286853790283203e-06,
2.5033950805664062e-06,
7.1227550506591797e-06]</div>
<div><br>
</div>
<div> DataRepresentation1 = Show()</div>
<div> DataRepresentation1.EdgeColor = [0.0,
0.0, 0.50000762951094835]</div>
<div>
DataRepresentation1.SelectionPointFieldDataArrayName
= 'Cf'</div>
<div> DataRepresentation1.ColorArrayName =
('POINT_DATA', 'Cf')</div>
<div>
DataRepresentation1.ScalarOpacityUnitDistance
= 0.033928799990483226</div>
<div> DataRepresentation1.ScaleFactor =
0.099998557567596444</div>
<div><br>
</div>
<div> a1_Cf_PVLookupTable =
GetLookupTableForArray( "Cf", 1,
RGBPoints=[-0.0099065126851201057,
0.23000000000000001, 0.29899999999999999,
0.754, 0.0097341262735426426,
0.86499999999999999, 0.86499999999999999,
0.86499999999999999, 0.029374765232205391,
0.70599999999999996, 0.016,
0.14999999999999999],
VectorMode='Magnitude', NanColor=[0.25, 0.0,
0.0], ColorSpace='Diverging',
ScalarRangeInitialized=1.0 )</div>
</div>
<div>
<div> a1_Cf_PiecewiseFunction =
CreatePiecewiseFunction(
Points=[-0.0099065126851201057, 0.0, 0.5,
0.0, 0.029374765232205391, 1.0, 0.5, 0.0] )</div>
<div><br>
</div>
<div>
DataRepresentation1.ScalarOpacityFunction =
a1_Cf_PiecewiseFunction</div>
<div> DataRepresentation1.LookupTable =
a1_Cf_PVLookupTable</div>
<div><br>
</div>
<div>
a1_Cf_PVLookupTable.ScalarOpacityFunction =
a1_Cf_PiecewiseFunction</div>
<div><br>
</div>
<div> a1_Cf_PVLookupTable.RGBPoints = [cfmin,
0.23000000000000001, 0.29899999999999999,
0.754, -0.002, 0.86499999999999999,
0.86499999999999999, 0.86499999999999999,
cfmax, 0.70599999999999996, 0.016,
0.14999999999999999]</div>
<div>
a1_Cf_PVLookupTable.ScalarOpacityFunction =
a1_Cf_PiecewiseFunction</div>
<div> a1_Cf_PVLookupTable.LockScalarRange = 1</div>
<div><br>
</div>
<div> RenderView1.CameraPosition =
[6.2286853790283203e-06,
2.5033950805664062e-06, 3.3460035306026636]</div>
<div> RenderView1.CameraFocalPoint =
[6.2286853790283203e-06,
2.5033950805664062e-06,
7.1227550506591797e-06]</div>
<div> RenderView1.CameraClippingRange =
[2.3175641399614015, 4.6486514901655163]</div>
<div> RenderView1.CameraParallelScale =
0.86600759519558368</div>
<div><br>
</div>
<div> Contour1 = Contour(
PointMergeMethod="Uniform Binning" )</div>
<div><br>
</div>
<div> Contour1.PointMergeMethod = "Uniform
Binning"</div>
<div> Contour1.ContourBy = ['POINTS', 'Cf']</div>
<div> Contour1.Isosurfaces =
[0.0097341262735426426]</div>
<div><br>
</div>
<div> DataRepresentation2 = Show()</div>
<div> DataRepresentation2.ColorArrayName =
('POINT_DATA', '')</div>
<div> DataRepresentation2.ScaleFactor =
0.099971860647201538</div>
<div>
DataRepresentation2.SelectionPointFieldDataArrayName
= 'Cf'</div>
<div> DataRepresentation2.EdgeColor = [0.0,
0.0, 0.50000762951094835]</div>
<div><br>
</div>
<div> DataRepresentation2.Representation =
'Wireframe'</div>
<div> DataRepresentation2.AmbientColor =
[0.0, 0.0, 0.0]</div>
<div><br>
</div>
<div> DataRepresentation1.Visibility = 0</div>
<div><br>
</div>
<div> RenderView1.CameraClippingRange =
[2.7470738944403328, 4.2397684828477349]</div>
<div><br>
</div>
<div> DataRepresentation1.Visibility = 1</div>
<div><br>
</div>
<div> RenderView1.CameraClippingRange =
[2.3175641399614015, 4.6486514901655163]</div>
<div><br>
</div>
<div> Q_VP_xmf = XDMFReader(
FileName=wdir+'/VPfield/XDMF/Q_VP'+index[i]+'.xmf'
)</div>
<div><br>
</div>
<div> Contour1.Isosurfaces = [0.0]</div>
<div><br>
</div>
<div> Q_VP_xmf.Sets = []</div>
<div> Q_VP_xmf.Grids = ['Eulerian Grid']</div>
<div> Q_VP_xmf.PointArrays = ['Q']</div>
<div><br>
</div>
<div> DataRepresentation3 = Show()</div>
<div> DataRepresentation3.EdgeColor = [0.0,
0.0, 0.50000762951094835]</div>
<div>
DataRepresentation3.SelectionPointFieldDataArrayName
= 'Q'</div>
<div>
DataRepresentation3.ScalarOpacityUnitDistance
= 0.0045778956118952439</div>
<div> DataRepresentation3.Representation =
'Outline'</div>
</div>
<div>
<div> DataRepresentation3.ScaleFactor =
0.25585924386978148</div>
<div><br>
</div>
<div> RenderView1.CameraClippingRange =
[0.30866115510384229, 6.1311800002592065]</div>
<div><br>
</div>
<div> Contour2 = Contour(
PointMergeMethod="Uniform Binning" )</div>
<div><br>
</div>
<div> Contour2.PointMergeMethod = "Uniform
Binning"</div>
<div> Contour2.ContourBy = ['POINTS', 'Q']</div>
<div> Contour2.Isosurfaces = [-348374.28125]</div>
<div><br>
</div>
<div> DataRepresentation4 = Show()</div>
<div> DataRepresentation4.ScaleFactor =
0.1348903000354767</div>
<div>
DataRepresentation4.SelectionPointFieldDataArrayName
= 'Normals'</div>
<div> DataRepresentation4.EdgeColor = [0.0,
0.0, 0.50000762951094835]</div>
<div><br>
</div>
<div> Contour2.Isosurfaces = [0.5]</div>
<div><br>
</div>
<div> DataRepresentation3.Visibility = 0</div>
<div><br>
</div>
<div> RenderView1.CameraClippingRange =
[0.46635874330917981, 5.5760844897764175]</div>
<div><br>
</div>
<div> RenderView1.CameraViewUp = [-1.0, 0.0,
0.0]</div>
<div> RenderView1.CameraPosition = [-5.0,
0.0, 5.0]</div>
<div> RenderView1.CameraClippingRange =
[3.0180305778980254, 7.5069525367021557]</div>
<div> RenderView1.CameraFocalPoint = [0.0,
0.0, 5.0]</div>
<div> RenderView1.CameraParallelScale =
1.565457167140391</div>
<div> RenderView1.CenterOfRotation = [0.0,
0.0, 0.0]</div>
<div><br>
</div>
<div> RenderView1.CameraFocalPoint = [0.0,
0.0, 0.5]</div>
<div> RenderView1.CameraClippingRange =
[0.86197012554183883, 4.2263653610655147]</div>
<div> RenderView1.CameraPosition = [0.0,
-2.5, 0.0]</div>
<div><br>
</div>
<div>
WriteImage(wdir+'/POSTPRO/IMAGES/Q_cf_VP'+index[i]+'.jpg')</div>
<div><br>
</div>
<div> Delete(DataRepresentation4)</div>
<div> Delete(Contour2)</div>
<div> Delete(DataRepresentation3)</div>
<div> Delete(Q_VP_xmf)</div>
<div><br>
</div>
<div> Delete(DataRepresentation2)</div>
<div> Delete(Contour1)</div>
<div> Delete(DataRepresentation1)</div>
<div> Delete(cf_VP_xmf)</div>
<div><br>
</div>
</div>
<div>The script was created by running an
interactive Paraview session and recording the
commands. I then modified it slightly to add
the loop and some variables.</div>
<div><br>
</div>
<div>I have been able to run the script
successfully on 1,2,16 and 32 cores
successfully and it loops twice. However when
I try to run on 64 cores I get the following
error message:</div>
<div><br>
</div>
<div>
<div>[node116:58372] 63 more processes have
sent help message help-mpi-common-cuda.txt /
dlopen failed</div>
<div>[node116:58372] Set MCA parameter
"orte_base_help_aggregate" to 0 to see all
help / error messages</div>
<div>terminate called after throwing an
instance of 'std::bad_alloc'</div>
<div> what(): std::bad_alloc</div>
<div>[node125:37180] *** Process received
signal ***</div>
<div>[node125:37180] Signal: Aborted (6)</div>
<div>[node125:37180] Signal code: (-6)</div>
<div>[node125:37180] [ 0]
/lib64/libpthread.so.0(+0xf710)
[0x2aaab28f3710]</div>
<div>[node125:37180] [ 1]
/lib64/libc.so.6(gsignal+0x35)
[0x2aaab479f625]</div>
<div>[node125:37180] [ 2]
/lib64/libc.so.6(abort+0x175)
[0x2aaab47a0e05]</div>
<div>[node125:37180] [ 3]
/usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d)
[0x2aaab408aa7d]</div>
<div>[node125:37180] [ 4]
/usr/lib64/libstdc++.so.6(+0xbcbd6)
[0x2aaab4088bd6]</div>
<div>[node125:37180] [ 5]
/usr/lib64/libstdc++.so.6(+0xbcc03)
[0x2aaab4088c03]</div>
<div>[node125:37180] [ 6]
/usr/lib64/libstdc++.so.6(+0xbcd22)
[0x2aaab4088d22]</div>
<div>[node125:37180] [ 7]
/usr/lib64/libstdc++.so.6(_Znwm+0x7d)
[0x2aaab408911d]</div>
<div>[node125:37180] [ 8]
/usr/lib64/libstdc++.so.6(_Znam+0x9)
[0x2aaab40891d9]</div>
<div>[node125:37180] [ 9]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData10ReadPointsEP12XdmfGeometryPiS2_+0x3a6)
[0x2aaab716310c]</div>
<div>[node125:37180] [10]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData21RequestStructuredGridEP8XdmfGrid+0xfb)
[0x2aaab716292f]</div>
<div>[node125:37180] [11]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData15ReadUniformDataEP8XdmfGrid+0x14c)
[0x2aaab71614ce]</div>
<div>[node125:37180] [12]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEP8XdmfGrid+0xd1)
[0x2aaab7160c91]</div>
<div>[node125:37180] [13]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEv+0x4f)
[0x2aaab7160a0f]</div>
<div>[node125:37180] [14]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader11RequestDataEP14vtkInformationPP20vtkInformationVectorS3_+0x31c)
[0x2aaab716daf0]</div>
<div>[node125:37180] [15]
/c1/apps/paraview/4.1.0/lib/libvtkIOLegacy-pv4.1.so.1(_ZN13vtkDataReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x62)
[0x2aaab0b3b9be]</div>
<div>[node125:37180] [16]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x76)
[0x2aaab716c702]</div>
<div>[node125:37180] [17]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN12vtkExecutive13CallAlgorithmEP14vtkInformationiPP20vtkInformationVectorS3_+0x8d)
[0x2aaab10c63d1]</div>
<div>[node125:37180] [18]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x76)
[0x2aaab10c0b1c]</div>
<div>[node125:37180] [19]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x468)
[0x2aaab10b9950]</div>
<div>[node125:37180] [20]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x522)
[0x2aaab10c0248]</div>
<div>[node125:37180] [21]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c)
[0x2aaab10e5ace]</div>
<div>[node125:37180] [22]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline15ForwardUpstreamEP14vtkInformation+0x2d6)
[0x2aaab10bbb6e]</div>
<div>[node125:37180] [23]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x48a)
[0x2aaab10c01b0]</div>
<div>[node125:37180] [24]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c)
[0x2aaab10e5ace]</div>
<div>[node125:37180] [25]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline10UpdateDataEi+0x2fb)
[0x2aaab10c0965]</div>
<div>[node125:37180] [26]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline6UpdateEi+0xf8)
[0x2aaab10e5d3e]</div>
<div>[node125:37180] [27]
/c1/apps/paraview/4.1.0/lib/libvtkPVServerImplementationCore-pv4.1.so.1(_ZN16vtkSISourceProxy14UpdatePipelineEidb+0x196)
[0x2aaaac77ef72]</div>
<div>[node125:37180] [28]
/c1/apps/paraview/4.1.0/lib/libvtkPVServerManagerApplication-pv4.1.so.1(_Z23vtkSISourceProxyCommandP26vtkClientServerInterpreterP13vtkObjectBasePKcRK21vtkClientServerStreamRS5_Pv+0x6ee)
[0x2aaaab7f27bd]</div>
<div>[node125:37180] [29]
/c1/apps/paraview/4.1.0/lib/libvtkClientServer-pv4.1.so.1(_ZN26vtkClientServerInterpreter19CallCommandFunctionEPKcP13vtkObjectBaseS1_RK21vtkClientServerStreamRS4_+0x343)
[0x2aaaafb57aa5]</div>
<div>[node125:37180] *** End of error message
***</div>
</div>
<div><br>
</div>
<div>I am afraid there is an issue with memory
leakage or improper memory allocation and
deallocation. At this point I do not know how
to proceed to debug it. Is there an obvious
problem in in the script and the way datasets
are deallocated at the end of each iteration?</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Nikos</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Powered by <a class="moz-txt-link-abbreviated" href="http://www.kitware.com">www.kitware.com</a>
Visit other Kitware open-source projects at <a class="moz-txt-link-freetext" href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a>
Please keep messages on-topic and check the ParaView Wiki at: <a class="moz-txt-link-freetext" href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a>
Search the list archives at: <a class="moz-txt-link-freetext" href="http://markmail.org/search/?q=ParaView">http://markmail.org/search/?q=ParaView</a>
Follow this link to subscribe/unsubscribe:
<a class="moz-txt-link-freetext" href="http://public.kitware.com/mailman/listinfo/paraview">http://public.kitware.com/mailman/listinfo/paraview</a>
</pre>
</blockquote>
<br>
</body>
</html>