[vtkusers] Memory leak vtkEnsightGoldReader in Python

Tue Mar 4 00:08:45 EST 2014

Therr is a python script inside the archive which is the same script that I
used on the large case.

Bernhard
On Mar 4, 2014 12:22 AM, "Bill Lorensen" <bill.lorensen at gmail.com> wrote:
>
> This is great. Can you send a complete python script that illustrates the
problem.?
>
> On Mar 3, 2014 2:47 PM, "Bernhard Righolt" <b.w.righolt at tudelft.nl> wrote:
> >
> > Hi Bill,
> >
> > I added a very small sample data set. I am not sure if the memory
problem I am encountering with my big data-set is measurable for this one.
> >
> > Anyhow, I would like to repeat that I am using Python 2.6 and VTK 5.10,
> >
> > Best regards,
> > Bernhard
> >
> >
> > On 3 March 2014 19:25, Bill Lorensen <bill.lorensen at gmail.com> wrote:
> >>
> >> The ensight readers could use more testing. Can you provide a "small"
> >> dataset and program that cause memory leaks?
> >>
> >> Bill
> >>
> >>
> >> On Mon, Mar 3, 2014 at 10:38 AM, Bernhard Righolt
> >> <b.w.righolt at tudelft.nl> wrote:
> >> > Dear all,
> >> >
> >> > A typical case file, looks as follows (ASCII).
> >> >
> >> > $ head -n 20 case.case
> >> > FORMAT
> >> > type: ensight gold
> >> >
> >> > GEOMETRY
> >> > model:        1     sliceCentre.000.mesh
> >> >
> >> > VARIABLE
> >> > vector per node:         1       U       sliceCentre.*****.U
> >> >
> >> > TIME
> >> > time set:                      1
> >> > number of steps:               11550
> >> > filename start number:         1
> >> > filename increment:            1
> >> > time values:
> >> > 0.002
> >> > 0.004
> >> > 0.006
> >> > 0.008
> >> > 0.01
> >> > $
> >> >
> >> > The mesh (snippet only), is as follows:
> >> >
> >> > $ head -n 10 sliceCentre.000.mesh
> >> > Ensight Geometry File
> >> > =====================
> >> > node id assign
> >> > element id assign
> >> > part
> >> >          1
> >> > sliceCentre.000.mesh
> >> > coordinates
> >> >     299028
> >> >  0.00000e+00
> >> > $ sed -n -e 897091,897098p sliceCentre.000.mesh
> >> >  0.00000e+00
> >> >  0.00000e+00
> >> >  0.00000e+00
> >> > tria3
> >> >     595264
> >> >     162106    162714    162799
> >> >     162799    162226    162106
> >> >     162714    162106    162021
> >> >
> >> >
> >> > And the corresponding datafile
> >> > $ head -n 10 sliceCentre.00001.U
> >> > vector
> >> > part
> >> >          1
> >> > coordinates
> >> >  5.33425e-04
> >> >  2.69515e-03
> >> > -1.65164e-03
> >> >  5.33425e-04
> >> >  1.39069e-04
> >> >  2.69515e-03
> >> >
> >> >
> >> > However, I doubt if something is wrong with my datafiles itself,
because I
> >> > can easily read them in Paraview for example, the only difference is
the
> >> > large amount of data-files, e.g (124GB in this case)
> >> >
> >> > Best regards,
> >> > Bernhard
> >> >
> >> >
> >> > On 27 February 2014 22:28, Berk Geveci <berk.geveci at kitware.com>
wrote:
> >> >>
> >> >> Nothing strikes me as obviously wrong in your script. Can you send
me
> >> >> your case file? I'll take a look.
> >> >>
> >> >> -berk
> >> >>
> >> >> On Thu, Feb 27, 2014 at 3:20 AM, Bernhard Righolt
> >> >> <b.w.righolt at tudelft.nl> wrote:
> >> >> > Dear all,
> >> >> >
> >> >> > Currently I am using a Python script to process information
stored in
> >> >> > the
> >> >> > EnSight Gold format. My Python (2.6) scipt uses VTK (5.10.0) to
process
> >> >> > the
> >> >> > file, where I used the vtkEnSightGoldReader for reading the data,
and
> >> >> > loop
> >> >> > over time steps.
> >> >> >
> >> >> > In principle this works fine for smaller datasets, however, for
large
> >> >> > datasets (i.e. 11k files of 12MB), I see the memory usage
(recorded via
> >> >> > top)
> >> >> > increasing with time while the process is running. This filling
of the
> >> >> > memory goes slow, but in some cases problems are inevitable.
Apparently,
> >> >> > I
> >> >> > am doing some things wrong in my script with respect to memory
> >> >> > management.
> >> >> >
> >> >> > I am using the following script (stripped obviously, but still
showing
> >> >> > this
> >> >> > issue)
> >> >> >
> >> >> > import vtk
> >> >> >
> >> >> > reader = vtk.vtkEnSightGoldReader()
> >> >> >
> >> >> > reader.SetCaseFileName("case.case")
> >> >> > reader.Update()
> >> >> >
> >> >> > # Get time values
> >> >> > timeset=reader.GetTimeSets()
> >> >> > time=timeset.GetItem(0)
> >> >> > timesteps=time.GetSize()
> >> >> >
> >> >> > #reader.ReleaseDataFlagOn()
> >> >> >
> >> >> > for j in range(timesteps):
> >> >> >     curTime=time.GetTuple(j)[0]
> >> >> >     print curTime
> >> >> >     reader.SetTimeValue(curTime)
> >> >> >     reader.Update()
> >> >> >
> >> >> >     #reader.RemoveAllInputs()
> >> >> >
> >> >> >
> >> >> > I tried a few things as you can see
> >> >> > - reader.ReleaseDataFlagOn(), without success
> >> >> > - reader.RemoveAllInputs(), did not get it to run
> >> >> > - running gc.collect at the end of the loop, following
> >> >> >
http://vtk.1045678.n5.nabble.com/VTK-memory-management-td5715661.html
> >> >> > - Using DeepCopy as suggested here:
> >> >> >
> >> >> >
http://vtk.1045678.n5.nabble.com/Memory-leak-in-VTK-Python-module-td1234002.html
> >> >> > However, I am already experiencing this without a call to
GetOutput
> >> >> >
> >> >> >
> >> >> > My question is, how can I properly unload/replace the data that is
> >> >> > stored in
> >> >> > the memory, instead of using more memory continuously?
> >> >> >
> >> >> > Best regards,
> >> >> > Bernhard
> >> >> >
> >> >> > _______________________________________________
> >> >> > Powered by www.kitware.com
> >> >> >
> >> >> > Visit other Kitware open-source projects at
> >> >> > http://www.kitware.com/opensource/opensource.html
> >> >> >
> >> >> > Please keep messages on-topic and check the VTK FAQ at:
> >> >> > http://www.vtk.org/Wiki/VTK_FAQ
> >> >> >
> >> >> > Follow this link to subscribe/unsubscribe:
> >> >> > http://www.vtk.org/mailman/listinfo/vtkusers
> >> >> >
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Powered by www.kitware.com
> >> >
> >> > Visit other Kitware open-source projects at
> >> > http://www.kitware.com/opensource/opensource.html
> >> >
> >> > Please keep messages on-topic and check the VTK FAQ at:
> >> > http://www.vtk.org/Wiki/VTK_FAQ
> >> >
> >> > Follow this link to subscribe/unsubscribe:
> >> > http://www.vtk.org/mailman/listinfo/vtkusers
> >> >
> >>
> >>
> >>
> >> --
> >> Unpaid intern in BillsBasement at noware dot com
> >
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140304/39d9faba/attachment.html>