[Paraview] Help with memory crash?

Wed Jan 7 21:17:22 EST 2015

So I did some more tests by running the script interactively. I used a
smaller grid first, 2002 x 20 x 1000 points and ran it only for 1
iteration. When Paraview loaded the memory usage on the server was 1.04GB.
After doing one iteration the memory usage spiked to 10.42GB and remained
like that when the script finished. What I found out is the Delete
statements make no difference at all, with or without them the memory usage
remained 10.42GB. Performing more iterations in the script actually worsens
this problem, after 4 iterations the memory used increased to 14.32GB. With
a larger grid of 2002 x 100 x 1000 points the memory after one iteration
increased from 10.42GB before the script to 45.25GB after the script. How
can I properly delete all datasets and clear the memory in Paraview?

On Wed, Jan 7, 2015 at 2:55 PM, Nikolaos Beratlis <nikos.beratlis at gmail.com>
wrote:

> I am trying to run a script on a cluster with Paraview in parallel. The
> cluster architecture is 16cores and 64GB per node. The script reads in two
> XMF files, one contains a grid of 2002x100x1000 points along with one
> variable. There is a loop to perform two iterations:
>
> try: paraview.simple
> except: from paraview.simple import *
> paraview.simple._DisableFirstRenderCameraReset()
>
> RenderView1 = GetRenderView()
> RenderView1.Background = [1.0, 1.0, 1.0]
> view = GetActiveView()
> view.ViewSize = [ 800, 500 ]
>
> RenderView1.CenterAxesVisibility = 0
> #RenderView1.OrientationAxesVisibility = 0
>
> #Define variables
> wdir =
> '/lustre/groups/balarasgrp/nikosb/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm'
> #wdir =
> '/Users/nikosb/colone/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm'
> #wdir = '/Users/nikosb/Research/Simulations/SPHERE/Re100/gridC1/run'
> Qval = 0.1
> cfmin = -0.001
> cfmax = 0.005
>
> index = [None]*9999
>
> m = 0
> for i in range(0, 9):
>   for j in range(0, 9):
>     for k in range(0, 9):
>       for l in range(0, 9):
>         index[m] = chr(i+48)+chr(j+48)+chr(k+48)+chr(l+48)
>         m = m+1
>
>
> for i in range(1,3):
>
>   cf_VP_xmf = XDMFReader(
> FileName=wdir+'/VPfield/XDMF/cf_VP'+index[i]+'.xmf' )
>
>   cf_VP_xmf.Sets = []
>   cf_VP_xmf.Grids = ['STL Grid']
>   cf_VP_xmf.PointArrays = ['Cf']
>
>   RenderView1 = GetRenderView()
>   RenderView1.CenterOfRotation = [6.2286853790283203e-06,
> 2.5033950805664062e-06, 7.1227550506591797e-06]
>
>   DataRepresentation1 = Show()
>   DataRepresentation1.EdgeColor = [0.0, 0.0, 0.50000762951094835]
>   DataRepresentation1.SelectionPointFieldDataArrayName = 'Cf'
>   DataRepresentation1.ColorArrayName = ('POINT_DATA', 'Cf')
>   DataRepresentation1.ScalarOpacityUnitDistance = 0.033928799990483226
>   DataRepresentation1.ScaleFactor = 0.099998557567596444
>
>   a1_Cf_PVLookupTable = GetLookupTableForArray( "Cf", 1,
> RGBPoints=[-0.0099065126851201057, 0.23000000000000001,
> 0.29899999999999999, 0.754, 0.0097341262735426426, 0.86499999999999999,
> 0.86499999999999999, 0.86499999999999999, 0.029374765232205391,
> 0.70599999999999996, 0.016, 0.14999999999999999], VectorMode='Magnitude',
> NanColor=[0.25, 0.0, 0.0], ColorSpace='Diverging',
> ScalarRangeInitialized=1.0 )
>   a1_Cf_PiecewiseFunction = CreatePiecewiseFunction(
> Points=[-0.0099065126851201057, 0.0, 0.5, 0.0, 0.029374765232205391, 1.0,
> 0.5, 0.0] )
>
>   DataRepresentation1.ScalarOpacityFunction = a1_Cf_PiecewiseFunction
>   DataRepresentation1.LookupTable = a1_Cf_PVLookupTable
>
>   a1_Cf_PVLookupTable.ScalarOpacityFunction = a1_Cf_PiecewiseFunction
>
>   a1_Cf_PVLookupTable.RGBPoints = [cfmin, 0.23000000000000001,
> 0.29899999999999999, 0.754, -0.002, 0.86499999999999999,
> 0.86499999999999999, 0.86499999999999999, cfmax, 0.70599999999999996,
> 0.016, 0.14999999999999999]
>   a1_Cf_PVLookupTable.ScalarOpacityFunction = a1_Cf_PiecewiseFunction
>   a1_Cf_PVLookupTable.LockScalarRange = 1
>
>   RenderView1.CameraPosition = [6.2286853790283203e-06,
> 2.5033950805664062e-06, 3.3460035306026636]
>   RenderView1.CameraFocalPoint = [6.2286853790283203e-06,
> 2.5033950805664062e-06, 7.1227550506591797e-06]
>   RenderView1.CameraClippingRange = [2.3175641399614015,
> 4.6486514901655163]
>   RenderView1.CameraParallelScale = 0.86600759519558368
>
>   Contour1 = Contour( PointMergeMethod="Uniform Binning" )
>
>   Contour1.PointMergeMethod = "Uniform Binning"
>   Contour1.ContourBy = ['POINTS', 'Cf']
>   Contour1.Isosurfaces = [0.0097341262735426426]
>
>   DataRepresentation2 = Show()
>   DataRepresentation2.ColorArrayName = ('POINT_DATA', '')
>   DataRepresentation2.ScaleFactor = 0.099971860647201538
>   DataRepresentation2.SelectionPointFieldDataArrayName = 'Cf'
>   DataRepresentation2.EdgeColor = [0.0, 0.0, 0.50000762951094835]
>
>   DataRepresentation2.Representation = 'Wireframe'
>   DataRepresentation2.AmbientColor = [0.0, 0.0, 0.0]
>
>   DataRepresentation1.Visibility = 0
>
>   RenderView1.CameraClippingRange = [2.7470738944403328,
> 4.2397684828477349]
>
>   DataRepresentation1.Visibility = 1
>
>   RenderView1.CameraClippingRange = [2.3175641399614015,
> 4.6486514901655163]
>
>   Q_VP_xmf = XDMFReader(
> FileName=wdir+'/VPfield/XDMF/Q_VP'+index[i]+'.xmf' )
>
>   Contour1.Isosurfaces = [0.0]
>
>   Q_VP_xmf.Sets = []
>   Q_VP_xmf.Grids = ['Eulerian Grid']
>   Q_VP_xmf.PointArrays = ['Q']
>
>   DataRepresentation3 = Show()
>   DataRepresentation3.EdgeColor = [0.0, 0.0, 0.50000762951094835]
>   DataRepresentation3.SelectionPointFieldDataArrayName = 'Q'
>   DataRepresentation3.ScalarOpacityUnitDistance = 0.0045778956118952439
>   DataRepresentation3.Representation = 'Outline'
>   DataRepresentation3.ScaleFactor = 0.25585924386978148
>
>   RenderView1.CameraClippingRange = [0.30866115510384229,
> 6.1311800002592065]
>
>   Contour2 = Contour( PointMergeMethod="Uniform Binning" )
>
>   Contour2.PointMergeMethod = "Uniform Binning"
>   Contour2.ContourBy = ['POINTS', 'Q']
>   Contour2.Isosurfaces = [-348374.28125]
>
>   DataRepresentation4 = Show()
>   DataRepresentation4.ScaleFactor = 0.1348903000354767
>   DataRepresentation4.SelectionPointFieldDataArrayName = 'Normals'
>   DataRepresentation4.EdgeColor = [0.0, 0.0, 0.50000762951094835]
>
>   Contour2.Isosurfaces = [0.5]
>
>   DataRepresentation3.Visibility = 0
>
>   RenderView1.CameraClippingRange = [0.46635874330917981,
> 5.5760844897764175]
>
>   RenderView1.CameraViewUp = [-1.0, 0.0, 0.0]
>   RenderView1.CameraPosition = [-5.0, 0.0, 5.0]
>   RenderView1.CameraClippingRange = [3.0180305778980254,
> 7.5069525367021557]
>   RenderView1.CameraFocalPoint = [0.0, 0.0, 5.0]
>   RenderView1.CameraParallelScale = 1.565457167140391
>   RenderView1.CenterOfRotation = [0.0, 0.0, 0.0]
>
>   RenderView1.CameraFocalPoint = [0.0, 0.0, 0.5]
>   RenderView1.CameraClippingRange = [0.86197012554183883,
> 4.2263653610655147]
>   RenderView1.CameraPosition = [0.0, -2.5, 0.0]
>
>   WriteImage(wdir+'/POSTPRO/IMAGES/Q_cf_VP'+index[i]+'.jpg')
>
>   Delete(DataRepresentation4)
>   Delete(Contour2)
>   Delete(DataRepresentation3)
>   Delete(Q_VP_xmf)
>
>   Delete(DataRepresentation2)
>   Delete(Contour1)
>   Delete(DataRepresentation1)
>   Delete(cf_VP_xmf)
>
> The script was created by running an interactive Paraview session and
> recording the commands. I then modified it slightly to add the loop and
> some variables.
>
> I have been able to run the script successfully on 1,2,16 and 32 cores
> successfully and it loops twice. However when I try to run on 64 cores I
> get the following error message:
>
> [node116:58372] 63 more processes have sent help message
> help-mpi-common-cuda.txt / dlopen failed
> [node116:58372] Set MCA parameter "orte_base_help_aggregate" to 0 to see
> all help / error messages
> terminate called after throwing an instance of 'std::bad_alloc'
>   what():  std::bad_alloc
> [node125:37180] *** Process received signal ***
> [node125:37180] Signal: Aborted (6)
> [node125:37180] Signal code:  (-6)
> [node125:37180] [ 0] /lib64/libpthread.so.0(+0xf710) [0x2aaab28f3710]
> [node125:37180] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2aaab479f625]
> [node125:37180] [ 2] /lib64/libc.so.6(abort+0x175) [0x2aaab47a0e05]
> [node125:37180] [ 3]
> /usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d)
> [0x2aaab408aa7d]
> [node125:37180] [ 4] /usr/lib64/libstdc++.so.6(+0xbcbd6) [0x2aaab4088bd6]
> [node125:37180] [ 5] /usr/lib64/libstdc++.so.6(+0xbcc03) [0x2aaab4088c03]
> [node125:37180] [ 6] /usr/lib64/libstdc++.so.6(+0xbcd22) [0x2aaab4088d22]
> [node125:37180] [ 7] /usr/lib64/libstdc++.so.6(_Znwm+0x7d) [0x2aaab408911d]
> [node125:37180] [ 8] /usr/lib64/libstdc++.so.6(_Znam+0x9) [0x2aaab40891d9]
> [node125:37180] [ 9]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData10ReadPointsEP12XdmfGeometryPiS2_+0x3a6)
> [0x2aaab716310c]
> [node125:37180] [10]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData21RequestStructuredGridEP8XdmfGrid+0xfb)
> [0x2aaab716292f]
> [node125:37180] [11]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData15ReadUniformDataEP8XdmfGrid+0x14c)
> [0x2aaab71614ce]
> [node125:37180] [12]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEP8XdmfGrid+0xd1)
> [0x2aaab7160c91]
> [node125:37180] [13]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEv+0x4f)
> [0x2aaab7160a0f]
> [node125:37180] [14]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader11RequestDataEP14vtkInformationPP20vtkInformationVectorS3_+0x31c)
> [0x2aaab716daf0]
> [node125:37180] [15]
> /c1/apps/paraview/4.1.0/lib/libvtkIOLegacy-pv4.1.so.1(_ZN13vtkDataReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x62)
> [0x2aaab0b3b9be]
> [node125:37180] [16]
> /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x76)
> [0x2aaab716c702]
> [node125:37180] [17]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN12vtkExecutive13CallAlgorithmEP14vtkInformationiPP20vtkInformationVectorS3_+0x8d)
> [0x2aaab10c63d1]
> [node125:37180] [18]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x76)
> [0x2aaab10c0b1c]
> [node125:37180] [19]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x468)
> [0x2aaab10b9950]
> [node125:37180] [20]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x522)
> [0x2aaab10c0248]
> [node125:37180] [21]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c)
> [0x2aaab10e5ace]
> [node125:37180] [22]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline15ForwardUpstreamEP14vtkInformation+0x2d6)
> [0x2aaab10bbb6e]
> [node125:37180] [23]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x48a)
> [0x2aaab10c01b0]
> [node125:37180] [24]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c)
> [0x2aaab10e5ace]
> [node125:37180] [25]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline10UpdateDataEi+0x2fb)
> [0x2aaab10c0965]
> [node125:37180] [26]
> /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline6UpdateEi+0xf8)
> [0x2aaab10e5d3e]
> [node125:37180] [27]
> /c1/apps/paraview/4.1.0/lib/libvtkPVServerImplementationCore-pv4.1.so.1(_ZN16vtkSISourceProxy14UpdatePipelineEidb+0x196)
> [0x2aaaac77ef72]
> [node125:37180] [28]
> /c1/apps/paraview/4.1.0/lib/libvtkPVServerManagerApplication-pv4.1.so.1(_Z23vtkSISourceProxyCommandP26vtkClientServerInterpreterP13vtkObjectBasePKcRK21vtkClientServerStreamRS5_Pv+0x6ee)
> [0x2aaaab7f27bd]
> [node125:37180] [29]
> /c1/apps/paraview/4.1.0/lib/libvtkClientServer-pv4.1.so.1(_ZN26vtkClientServerInterpreter19CallCommandFunctionEPKcP13vtkObjectBaseS1_RK21vtkClientServerStreamRS4_+0x343)
> [0x2aaaafb57aa5]
> [node125:37180] *** End of error message ***
>
> I am afraid there is an issue with memory leakage or improper memory
> allocation and deallocation. At this point I do not know how to proceed to
> debug it. Is there an obvious problem in in the script and the way datasets
> are deallocated at the end of each iteration?
>
> Regards,
>
> Nikos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20150107/94612ce8/attachment.html>