[Paraview] Memory overload and I don't know what to delete (despite looking in the archives)
Tim De Coster
tim91decoster at gmail.com
Tue Jan 19 16:33:49 EST 2016
Sir,
Thank you very much for answering so quickly. I went with the first option
and tried the Delete() statements again, but still keep getting errors, be
it different ones. Specifically I have put the following four statements
right after Render() in my code:
Delete(VoltParaviewMovie)
Delete(AnimationScene1)
Delete(RenderView1)
Delete(DataRepresentation1)
After executing I get the following error:
Traceback (most recent call last):
File "./RunParaviewMovieMaker.py", line 99, in <module>
main()
File "./RunParaviewMovieMaker.py", line 21, in main
create_movie(filename3)
File "./RunParaviewMovieMaker.py", line 91, in create_movie
Delete(DataRepresentation1)
File "/usr/lib/python2.7/dist-packages/paraview/simple.py", line 482, in
Delete
servermanager.UnRegister(proxy)
File "/usr/lib/python2.7/dist-packages/paraview/servermanager.py", line
2764, in UnRegister
raise RuntimeError, "UnRegistration error."
RuntimeError: UnRegistration error.
*** Error in `/usr/bin/python': corrupted double-linked list:
0x0000000004633310 ***
Aborted (core dumped)
Upon removing the last line, and just working with 3 Delete() statements, I
also get an error, namely:
Traceback (most recent call last):
File "./RunParaviewMovieMaker.py", line 98, in <module>
main()
File "./RunParaviewMovieMaker.py", line 21, in main
create_movie(filename3)
File "./RunParaviewMovieMaker.py", line 44, in create_movie
AnimationScene1 = GetAnimationScene()
File "/usr/lib/python2.7/dist-packages/paraview/simple.py", line 794, in
GetAnimationScene
raise servermanager.MissingProxy, "Could not locate global
AnimationScene."
paraview.servermanager.MissingProxy: Could not locate global AnimationScene.
*** Error in `/usr/bin/python': double free or corruption (!prev):
0x0000000002908e30 ***
Aborted (core dumped)
Do you have any idea on how to resolve these errors? I do especially not
understand what the 'proxy' is doing in my errors.
If not, I might opt for the second option with ImageMagick (although I
would still prefer to resolve my current issues).
With kind regards,
Tim
2016-01-19 18:28 GMT+01:00 Tim Gallagher <tim.gallagher at gatech.edu>:
> Your delete line is
>
> Delete(Animationscene1)
>
> but your variable is actually called AnimationScene1 (camel-case
> lettering). Python is case-sensitive, so that's why you are getting the
> error you are showing.
>
> That said, we do two things to loop over everything. We can Delete() every
> object, but that can be tedious in complicated pipelines. Alternatively, we
> have a driver script that executes pvpython on a trace script and passes in
> a number to indicate which file should be rendered. Our trace script then
> dumps a single PNG image and we stitch them together to make a movie
> afterwards using ImageMagick. Since each image is rendered in an entirely
> new instance of pvpython, there are no memory issues. An example driver
> script is below. We've also made our driver script parallel so it splits
> the range of data files into chunks for each processor, and then pvpython
> is called multiple times in parallel to render the files.
>
> Tim
>
> import subprocess
>
> start = 0
> end = 100
> stride = 1
>
> for n in range(start, end + 1, stride):
> print "Working on file: ", n
> subprocess.call(['/data4/pv4.1Install-OS/bin/pvpython',
> 'jicf_trace.py', '%i' % (n)])
>
> ------------------------------
> *From: *"Tim De Coster" <tim91decoster at gmail.com>
> *To: *paraview at paraview.org
> *Sent: *Tuesday, January 19, 2016 10:03:01 AM
> *Subject: *[Paraview] Memory overload and I don't know what to delete
> (despite looking in the archives)
>
>
> Hi all,
>
> I am stuck with a problem regarding ParaView (version 4.0.1 64-bit),
> running on Ubuntu 14.04. It seems that when I run a python script, my
> memory gets overloaded. I looked already in the archives and it seems that
> I need to delete some data using the command Delete(). When I try this
> however, I get error messages saying that the things I want to delete are
> not possible
> ( File "./RunParaviewMovieMaker.py", line 89, in create_movie
> Delete(Animationscene1)
> NameError: global name 'Animationscene1' is not defined
> *** Error in `/usr/bin/python': corrupted double-linked list:
> 0x0000000002b52ca0 ***
> ).
> Let me first explain my problem in more detail:
>
> I have a lot of data that I generated with another script (about 3TB and
> still increasing), but now I need to convert these data into movies. When I
> run my loop over different folders, the first few movies are generated in
> less than 30s each, but then gradually the time per movie starts increasing
> and my memory gets overloaded. The script I am running is (which I simply
> got from using the trace call for python in ParaView):
>
> #!/usr/bin/python
>
> from paraview.simple import *
>
> def main():
>
> some huge for-loop generating names of maps where the .vtk files are
> stored. In this for loop I use the command:
> create_movie(filename3)
>
> def create_movie(mapname):
>
> vtkfolder = '.'
> number_of_frames = 2000
>
> vtknames = []
> for i in range (0, number_of_frames - 1):
> vtknames.append('{1}/{2}/VoltParaviewMovie{0:0=4d}.vtk'.format(i,
> vtkfolder, mapname))
>
> VoltParaviewMovie = LegacyVTKReader(FileNames = vtknames)
>
> AnimationScene1 = GetAnimationScene()
> AnimationScene1 = GetAnimationScene()
> AnimationScene1.EndTime = 1999.0
> AnimationScene1.PlayMode = 'Snap To TimeSteps'
>
> AnimationScene1.EndTime = 1999.0
> AnimationScene1.PlayMode = 'Snap To TimeSteps'
>
> RenderView1 = GetRenderView()
> a1_Voltage_PVLookupTable = GetLookupTableForArray( "Voltage", 1,
> RGBPoints=[-85.52999877929688, 0.23, 0.299, 0.754, 50.0, 0.706, 0.016,
> 0.15], VectorMode='Magnitude', NanColor=[0.25, 0.0, 0.0],
> ColorSpace='Diverging', ScalarRangeInitialized=1.0, AllowDuplicateScalars=1
> )
>
> a1_Voltage_PiecewiseFunction = CreatePiecewiseFunction( Points=[0.0,
> 0.0, 0.5, 0.0, 1.0, 1.0, 0.5, 0.0] )
>
> DataRepresentation1 = Show()
> DataRepresentation1.EdgeColor = [0.0, 0.0, 0.5000076295109483]
> DataRepresentation1.SelectionPointFieldDataArrayName = 'Voltage'
> DataRepresentation1.ScalarOpacityFunction =
> a1_Voltage_PiecewiseFunction
> DataRepresentation1.ColorArrayName = ('POINT_DATA', 'Voltage')
> DataRepresentation1.ScalarOpacityUnitDistance = 7.108580809929175
> DataRepresentation1.LookupTable = a1_Voltage_PVLookupTable
> DataRepresentation1.Representation = 'Slice'
> DataRepresentation1.ScaleFactor = 0.0
>
> RenderView1.CameraFocalPoint = [63.5, 63.5, 0.0]
> RenderView1.CameraPosition = [63.5, 63.5, 10000.0]
> RenderView1.InteractionMode = '2D'
> RenderView1.CenterOfRotation = [63.5, 63.5, 0.0]
>
> a1_Voltage_PVLookupTable.ScalarOpacityFunction =
> a1_Voltage_PiecewiseFunction
>
> RenderView1.CameraPosition = [63.5, 63.5, 346.97045256124744]
> RenderView1.CameraClippingRange = [343.50074803563496,
> 352.1750093496662]
> RenderView1.CameraParallelScale = 89.80256121069154
>
> newmapname = ''
> fragments = mapname.split('.')
> for fragment in fragments:
> newmapname +=fragment
>
> WriteAnimation('{0}/{1}.avi'.format(vtkfolder, newmapname),
> Magnification=1, Quality=2, FrameRate=15.000000)
>
> Render()
>
> print('{0} has been produced.'.format(newmapname))
>
>
> if __name__ == '__main__':
> main()
>
> Would any one of you be able to point out to me what I am doing wrong? Or
> which objects/structures I should delete (in the archives the problem
> always appeared to be that a writer needed to be deleted, but I don't have
> such an object) such that I basically start over each time I create a new
> movie and hence don't use that much memory?
>
> With kind regards,
> Tim
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Search the list archives at: http://markmail.org/search/?q=ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/paraview
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/paraview/attachments/20160119/56425b09/attachment.html>
More information about the ParaView
mailing list