[Paraview] Memory overload and I don't know what to delete (despite looking in the archives)

[Tim Gallagher]

Your delete line is 

Delete(Animationscene1) 

but your variable is actually called AnimationScene1 (camel-case lettering). Python is case-sensitive, so that's why you are getting the error you are showing. 

That said, we do two things to loop over everything. We can Delete() every object, but that can be tedious in complicated pipelines. Alternatively, we have a driver script that executes pvpython on a trace script and passes in a number to indicate which file should be rendered. Our trace script then dumps a single PNG image and we stitch them together to make a movie afterwards using ImageMagick. Since each image is rendered in an entirely new instance of pvpython, there are no memory issues. An example driver script is below. We've also made our driver script parallel so it splits the range of data files into chunks for each processor, and then pvpython is called multiple times in parallel to render the files. 

Tim 

import subprocess 

start = 0 
end = 100 
stride = 1 

for n in range(start, end + 1, stride): 
print "Working on file: ", n 
subprocess.call(['/data4/pv4.1Install-OS/bin/pvpython', 'jicf_trace.py', '%i' % (n)]) 

----- Original Message -----

From: "Tim De Coster" <tim91decoster at gmail.com> 
To: paraview at paraview.org 
Sent: Tuesday, January 19, 2016 10:03:01 AM 
Subject: [Paraview] Memory overload and I don't know what to delete (despite looking in the archives) 







Hi all, 

I am stuck with a problem regarding ParaView (version 4.0.1 64-bit), running on Ubuntu 14.04. It seems that when I run a python script, my memory gets overloaded. I looked already in the archives and it seems that I need to delete some data using the command Delete(). When I try this however, I get error messages saying that the things I want to delete are not possible 
( File "./RunParaviewMovieMaker.py", line 89, in create_movie 
Delete(Animationscene1) 
NameError: global name 'Animationscene1' is not defined 
*** Error in `/usr/bin/python': corrupted double-linked list: 0x0000000002b52ca0 *** 
). 
Let me first explain my problem in more detail: 

I have a lot of data that I generated with another script (about 3TB and still increasing), but now I need to convert these data into movies. When I run my loop over different folders, the first few movies are generated in less than 30s each, but then gradually the time per movie starts increasing and my memory gets overloaded. The script I am running is (which I simply got from using the trace call for python in ParaView): 

#!/usr/bin/python 

from paraview.simple import * 

def main(): 

some huge for-loop generating names of maps where the .vtk files are stored. In this for loop I use the command: 
create_movie(filename3) 

def create_movie(mapname): 

vtkfolder = '.' 
number_of_frames = 2000 

vtknames = [] 
for i in range (0, number_of_frames - 1): 
vtknames.append('{1}/{2}/VoltParaviewMovie{0:0=4d}.vtk'.format(i, vtkfolder, mapname)) 

VoltParaviewMovie = LegacyVTKReader(FileNames = vtknames) 

AnimationScene1 = GetAnimationScene() 
AnimationScene1 = GetAnimationScene() 
AnimationScene1.EndTime = 1999.0 
AnimationScene1.PlayMode = 'Snap To TimeSteps' 

AnimationScene1.EndTime = 1999.0 
AnimationScene1.PlayMode = 'Snap To TimeSteps' 

RenderView1 = GetRenderView() 
a1_Voltage_PVLookupTable = GetLookupTableForArray( "Voltage", 1, RGBPoints=[-85.52999877929688, 0.23, 0.299, 0.754, 50.0, 0.706, 0.016, 0.15], VectorMode='Magnitude', NanColor=[0.25, 0.0, 0.0], ColorSpace='Diverging', ScalarRangeInitialized=1.0, AllowDuplicateScalars=1 ) 

a1_Voltage_PiecewiseFunction = CreatePiecewiseFunction( Points=[0.0, 0.0, 0.5, 0.0, 1.0, 1.0, 0.5, 0.0] ) 

DataRepresentation1 = Show() 
DataRepresentation1.EdgeColor = [0.0, 0.0, 0.5000076295109483] 
DataRepresentation1.SelectionPointFieldDataArrayName = 'Voltage' 
DataRepresentation1.ScalarOpacityFunction = a1_Voltage_PiecewiseFunction 
DataRepresentation1.ColorArrayName = ('POINT_DATA', 'Voltage') 
DataRepresentation1.ScalarOpacityUnitDistance = 7.108580809929175 
DataRepresentation1.LookupTable = a1_Voltage_PVLookupTable 
DataRepresentation1.Representation = 'Slice' 
DataRepresentation1.ScaleFactor = 0.0 

RenderView1.CameraFocalPoint = [63.5, 63.5, 0.0] 
RenderView1.CameraPosition = [63.5, 63.5, 10000.0] 
RenderView1.InteractionMode = '2D' 
RenderView1.CenterOfRotation = [63.5, 63.5, 0.0] 

a1_Voltage_PVLookupTable.ScalarOpacityFunction = a1_Voltage_PiecewiseFunction 

RenderView1.CameraPosition = [63.5, 63.5, 346.97045256124744] 
RenderView1.CameraClippingRange = [343.50074803563496, 352.1750093496662] 
RenderView1.CameraParallelScale = 89.80256121069154 

newmapname = '' 
fragments = mapname.split('.') 
for fragment in fragments: 
newmapname +=fragment 

WriteAnimation('{0}/{1}.avi'.format(vtkfolder, newmapname), Magnification=1, Quality=2, FrameRate=15.000000) 

Render() 

print('{0} has been produced.'.format(newmapname)) 


if __name__ == '__main__': 
main() 

Would any one of you be able to point out to me what I am doing wrong? Or which objects/structures I should delete (in the archives the problem always appeared to be that a writer needed to be deleted, but I don't have such an object) such that I basically start over each time I create a new movie and hence don't use that much memory? 

With kind regards, 
Tim 
_______________________________________________ 
Powered by www.kitware.com 

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html 

Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView 

Search the list archives at: http://markmail.org/search/?q=ParaView 

Follow this link to subscribe/unsubscribe: 
http://public.kitware.com/mailman/listinfo/paraview 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20160119/84bd4b3d/attachment.html>