[vtkusers] Memory leak vtkEnsightGoldReader in Python

[Bill Lorensen]

This is great. Can you send a complete python script that illustrates the
problem.?
On Mar 3, 2014 2:47 PM, "Bernhard Righolt" <b.w.righolt at tudelft.nl> wrote:
>
> Hi Bill,
>
> I added a very small sample data set. I am not sure if the memory problem
I am encountering with my big data-set is measurable for this one.
>
> Anyhow, I would like to repeat that I am using Python 2.6 and VTK 5.10,
>
> Best regards,
> Bernhard
>
>
> On 3 March 2014 19:25, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>
>> The ensight readers could use more testing. Can you provide a "small"
>> dataset and program that cause memory leaks?
>>
>> Bill
>>
>>
>> On Mon, Mar 3, 2014 at 10:38 AM, Bernhard Righolt
>> <b.w.righolt at tudelft.nl> wrote:
>> > Dear all,
>> >
>> > A typical case file, looks as follows (ASCII).
>> >
>> > $ head -n 20 case.case
>> > FORMAT
>> > type: ensight gold
>> >
>> > GEOMETRY
>> > model:        1     sliceCentre.000.mesh
>> >
>> > VARIABLE
>> > vector per node:         1       U       sliceCentre.*****.U
>> >
>> > TIME
>> > time set:                      1
>> > number of steps:               11550
>> > filename start number:         1
>> > filename increment:            1
>> > time values:
>> > 0.002
>> > 0.004
>> > 0.006
>> > 0.008
>> > 0.01
>> > $
>> >
>> > The mesh (snippet only), is as follows:
>> >
>> > $ head -n 10 sliceCentre.000.mesh
>> > Ensight Geometry File
>> > =====================
>> > node id assign
>> > element id assign
>> > part
>> >          1
>> > sliceCentre.000.mesh
>> > coordinates
>> >     299028
>> >  0.00000e+00
>> > $ sed -n -e 897091,897098p sliceCentre.000.mesh
>> >  0.00000e+00
>> >  0.00000e+00
>> >  0.00000e+00
>> > tria3
>> >     595264
>> >     162106    162714    162799
>> >     162799    162226    162106
>> >     162714    162106    162021
>> >
>> >
>> > And the corresponding datafile
>> > $ head -n 10 sliceCentre.00001.U
>> > vector
>> > part
>> >          1
>> > coordinates
>> >  5.33425e-04
>> >  2.69515e-03
>> > -1.65164e-03
>> >  5.33425e-04
>> >  1.39069e-04
>> >  2.69515e-03
>> >
>> >
>> > However, I doubt if something is wrong with my datafiles itself,
because I
>> > can easily read them in Paraview for example, the only difference is
the
>> > large amount of data-files, e.g (124GB in this case)
>> >
>> > Best regards,
>> > Bernhard
>> >
>> >
>> > On 27 February 2014 22:28, Berk Geveci <berk.geveci at kitware.com> wrote:
>> >>
>> >> Nothing strikes me as obviously wrong in your script. Can you send me
>> >> your case file? I'll take a look.
>> >>
>> >> -berk
>> >>
>> >> On Thu, Feb 27, 2014 at 3:20 AM, Bernhard Righolt
>> >> <b.w.righolt at tudelft.nl> wrote:
>> >> > Dear all,
>> >> >
>> >> > Currently I am using a Python script to process information stored
in
>> >> > the
>> >> > EnSight Gold format. My Python (2.6) scipt uses VTK (5.10.0) to
process
>> >> > the
>> >> > file, where I used the vtkEnSightGoldReader for reading the data,
and
>> >> > loop
>> >> > over time steps.
>> >> >
>> >> > In principle this works fine for smaller datasets, however, for
large
>> >> > datasets (i.e. 11k files of 12MB), I see the memory usage (recorded
via
>> >> > top)
>> >> > increasing with time while the process is running. This filling of
the
>> >> > memory goes slow, but in some cases problems are inevitable.
Apparently,
>> >> > I
>> >> > am doing some things wrong in my script with respect to memory
>> >> > management.
>> >> >
>> >> > I am using the following script (stripped obviously, but still
showing
>> >> > this
>> >> > issue)
>> >> >
>> >> > import vtk
>> >> >
>> >> > reader = vtk.vtkEnSightGoldReader()
>> >> >
>> >> > reader.SetCaseFileName("case.case")
>> >> > reader.Update()
>> >> >
>> >> > # Get time values
>> >> > timeset=reader.GetTimeSets()
>> >> > time=timeset.GetItem(0)
>> >> > timesteps=time.GetSize()
>> >> >
>> >> > #reader.ReleaseDataFlagOn()
>> >> >
>> >> > for j in range(timesteps):
>> >> >     curTime=time.GetTuple(j)[0]
>> >> >     print curTime
>> >> >     reader.SetTimeValue(curTime)
>> >> >     reader.Update()
>> >> >
>> >> >     #reader.RemoveAllInputs()
>> >> >
>> >> >
>> >> > I tried a few things as you can see
>> >> > - reader.ReleaseDataFlagOn(), without success
>> >> > - reader.RemoveAllInputs(), did not get it to run
>> >> > - running gc.collect at the end of the loop, following
>> >> >
http://vtk.1045678.n5.nabble.com/VTK-memory-management-td5715661.html
>> >> > - Using DeepCopy as suggested here:
>> >> >
>> >> >
http://vtk.1045678.n5.nabble.com/Memory-leak-in-VTK-Python-module-td1234002.html
>> >> > However, I am already experiencing this without a call to GetOutput
>> >> >
>> >> >
>> >> > My question is, how can I properly unload/replace the data that is
>> >> > stored in
>> >> > the memory, instead of using more memory continuously?
>> >> >
>> >> > Best regards,
>> >> > Bernhard
>> >> >
>> >> > _______________________________________________
>> >> > Powered by www.kitware.com
>> >> >
>> >> > Visit other Kitware open-source projects at
>> >> > http://www.kitware.com/opensource/opensource.html
>> >> >
>> >> > Please keep messages on-topic and check the VTK FAQ at:
>> >> > http://www.vtk.org/Wiki/VTK_FAQ
>> >> >
>> >> > Follow this link to subscribe/unsubscribe:
>> >> > http://www.vtk.org/mailman/listinfo/vtkusers
>> >> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html
>> >
>> > Please keep messages on-topic and check the VTK FAQ at:
>> > http://www.vtk.org/Wiki/VTK_FAQ
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://www.vtk.org/mailman/listinfo/vtkusers
>> >
>>
>>
>>
>> --
>> Unpaid intern in BillsBasement at noware dot com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140303/30651ef2/attachment.html>