[vtkusers] Memory leak vtkEnsightGoldReader in Python
Bernhard Righolt
b.w.righolt at tudelft.nl
Tue Mar 4 15:10:55 EST 2014
Dear Bill,
I am a C++ speaker, so I don't mind to test your code on my current large
dataset. Can you maybe share your code, such that I can test it? I've never
used VTK within C++, if it solves the issues I am experiencing with Python,
I am very willing to switch. I hope it is not too complicated to link the
libraries correctly.
Bernhard
On 4 March 2014 21:05, Bill Lorensen <bill.lorensen at gmail.com> wrote:
> Bernhard,
>
> Thanks for your patience. Since I'm not a python person, I converted
> your program to C++. I ran valgrind and found no memory leaks.
>
> Perhaps the small dataset you provided does not exercise the code that
> is leaking.
>
> Can you possible place a dataset that you know has leaksat some accessible
> site?
>
> I will say, that I don't think any of the current ensight tests access
> multiple time steps. Your data and test script will be a good starting
> point for such a test.
>
> Bill
>
> On Tue, Mar 4, 2014 at 12:08 AM, Bernhard Righolt
> <b.w.righolt at tudelft.nl> wrote:
> > Therr is a python script inside the archive which is the same script
> that I
> > used on the large case.
> >
> > Bernhard
> >
> >
> > On Mar 4, 2014 12:22 AM, "Bill Lorensen" <bill.lorensen at gmail.com>
> wrote:
> >>
> >> This is great. Can you send a complete python script that illustrates
> the
> >> problem.?
> >>
> >> On Mar 3, 2014 2:47 PM, "Bernhard Righolt" <b.w.righolt at tudelft.nl>
> wrote:
> >> >
> >> > Hi Bill,
> >> >
> >> > I added a very small sample data set. I am not sure if the memory
> >> > problem I am encountering with my big data-set is measurable for this
> one.
> >> >
> >> > Anyhow, I would like to repeat that I am using Python 2.6 and VTK
> 5.10,
> >> >
> >> > Best regards,
> >> > Bernhard
> >> >
> >> >
> >> > On 3 March 2014 19:25, Bill Lorensen <bill.lorensen at gmail.com> wrote:
> >> >>
> >> >> The ensight readers could use more testing. Can you provide a "small"
> >> >> dataset and program that cause memory leaks?
> >> >>
> >> >> Bill
> >> >>
> >> >>
> >> >> On Mon, Mar 3, 2014 at 10:38 AM, Bernhard Righolt
> >> >> <b.w.righolt at tudelft.nl> wrote:
> >> >> > Dear all,
> >> >> >
> >> >> > A typical case file, looks as follows (ASCII).
> >> >> >
> >> >> > $ head -n 20 case.case
> >> >> > FORMAT
> >> >> > type: ensight gold
> >> >> >
> >> >> > GEOMETRY
> >> >> > model: 1 sliceCentre.000.mesh
> >> >> >
> >> >> > VARIABLE
> >> >> > vector per node: 1 U sliceCentre.*****.U
> >> >> >
> >> >> > TIME
> >> >> > time set: 1
> >> >> > number of steps: 11550
> >> >> > filename start number: 1
> >> >> > filename increment: 1
> >> >> > time values:
> >> >> > 0.002
> >> >> > 0.004
> >> >> > 0.006
> >> >> > 0.008
> >> >> > 0.01
> >> >> > $
> >> >> >
> >> >> > The mesh (snippet only), is as follows:
> >> >> >
> >> >> > $ head -n 10 sliceCentre.000.mesh
> >> >> > Ensight Geometry File
> >> >> > =====================
> >> >> > node id assign
> >> >> > element id assign
> >> >> > part
> >> >> > 1
> >> >> > sliceCentre.000.mesh
> >> >> > coordinates
> >> >> > 299028
> >> >> > 0.00000e+00
> >> >> > $ sed -n -e 897091,897098p sliceCentre.000.mesh
> >> >> > 0.00000e+00
> >> >> > 0.00000e+00
> >> >> > 0.00000e+00
> >> >> > tria3
> >> >> > 595264
> >> >> > 162106 162714 162799
> >> >> > 162799 162226 162106
> >> >> > 162714 162106 162021
> >> >> >
> >> >> >
> >> >> > And the corresponding datafile
> >> >> > $ head -n 10 sliceCentre.00001.U
> >> >> > vector
> >> >> > part
> >> >> > 1
> >> >> > coordinates
> >> >> > 5.33425e-04
> >> >> > 2.69515e-03
> >> >> > -1.65164e-03
> >> >> > 5.33425e-04
> >> >> > 1.39069e-04
> >> >> > 2.69515e-03
> >> >> >
> >> >> >
> >> >> > However, I doubt if something is wrong with my datafiles itself,
> >> >> > because I
> >> >> > can easily read them in Paraview for example, the only difference
> is
> >> >> > the
> >> >> > large amount of data-files, e.g (124GB in this case)
> >> >> >
> >> >> > Best regards,
> >> >> > Bernhard
> >> >> >
> >> >> >
> >> >> > On 27 February 2014 22:28, Berk Geveci <berk.geveci at kitware.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Nothing strikes me as obviously wrong in your script. Can you send
> >> >> >> me
> >> >> >> your case file? I'll take a look.
> >> >> >>
> >> >> >> -berk
> >> >> >>
> >> >> >> On Thu, Feb 27, 2014 at 3:20 AM, Bernhard Righolt
> >> >> >> <b.w.righolt at tudelft.nl> wrote:
> >> >> >> > Dear all,
> >> >> >> >
> >> >> >> > Currently I am using a Python script to process information
> stored
> >> >> >> > in
> >> >> >> > the
> >> >> >> > EnSight Gold format. My Python (2.6) scipt uses VTK (5.10.0) to
> >> >> >> > process
> >> >> >> > the
> >> >> >> > file, where I used the vtkEnSightGoldReader for reading the
> data,
> >> >> >> > and
> >> >> >> > loop
> >> >> >> > over time steps.
> >> >> >> >
> >> >> >> > In principle this works fine for smaller datasets, however, for
> >> >> >> > large
> >> >> >> > datasets (i.e. 11k files of 12MB), I see the memory usage
> >> >> >> > (recorded via
> >> >> >> > top)
> >> >> >> > increasing with time while the process is running. This filling
> of
> >> >> >> > the
> >> >> >> > memory goes slow, but in some cases problems are inevitable.
> >> >> >> > Apparently,
> >> >> >> > I
> >> >> >> > am doing some things wrong in my script with respect to memory
> >> >> >> > management.
> >> >> >> >
> >> >> >> > I am using the following script (stripped obviously, but still
> >> >> >> > showing
> >> >> >> > this
> >> >> >> > issue)
> >> >> >> >
> >> >> >> > import vtk
> >> >> >> >
> >> >> >> > reader = vtk.vtkEnSightGoldReader()
> >> >> >> >
> >> >> >> > reader.SetCaseFileName("case.case")
> >> >> >> > reader.Update()
> >> >> >> >
> >> >> >> > # Get time values
> >> >> >> > timeset=reader.GetTimeSets()
> >> >> >> > time=timeset.GetItem(0)
> >> >> >> > timesteps=time.GetSize()
> >> >> >> >
> >> >> >> > #reader.ReleaseDataFlagOn()
> >> >> >> >
> >> >> >> > for j in range(timesteps):
> >> >> >> > curTime=time.GetTuple(j)[0]
> >> >> >> > print curTime
> >> >> >> > reader.SetTimeValue(curTime)
> >> >> >> > reader.Update()
> >> >> >> >
> >> >> >> > #reader.RemoveAllInputs()
> >> >> >> >
> >> >> >> >
> >> >> >> > I tried a few things as you can see
> >> >> >> > - reader.ReleaseDataFlagOn(), without success
> >> >> >> > - reader.RemoveAllInputs(), did not get it to run
> >> >> >> > - running gc.collect at the end of the loop, following
> >> >> >> >
> >> >> >> >
> http://vtk.1045678.n5.nabble.com/VTK-memory-management-td5715661.html
> >> >> >> > - Using DeepCopy as suggested here:
> >> >> >> >
> >> >> >> >
> >> >> >> >
> http://vtk.1045678.n5.nabble.com/Memory-leak-in-VTK-Python-module-td1234002.html
> >> >> >> > However, I am already experiencing this without a call to
> >> >> >> > GetOutput
> >> >> >> >
> >> >> >> >
> >> >> >> > My question is, how can I properly unload/replace the data that
> is
> >> >> >> > stored in
> >> >> >> > the memory, instead of using more memory continuously?
> >> >> >> >
> >> >> >> > Best regards,
> >> >> >> > Bernhard
> >> >> >> >
> >> >> >> > _______________________________________________
> >> >> >> > Powered by www.kitware.com
> >> >> >> >
> >> >> >> > Visit other Kitware open-source projects at
> >> >> >> > http://www.kitware.com/opensource/opensource.html
> >> >> >> >
> >> >> >> > Please keep messages on-topic and check the VTK FAQ at:
> >> >> >> > http://www.vtk.org/Wiki/VTK_FAQ
> >> >> >> >
> >> >> >> > Follow this link to subscribe/unsubscribe:
> >> >> >> > http://www.vtk.org/mailman/listinfo/vtkusers
> >> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > Powered by www.kitware.com
> >> >> >
> >> >> > Visit other Kitware open-source projects at
> >> >> > http://www.kitware.com/opensource/opensource.html
> >> >> >
> >> >> > Please keep messages on-topic and check the VTK FAQ at:
> >> >> > http://www.vtk.org/Wiki/VTK_FAQ
> >> >> >
> >> >> > Follow this link to subscribe/unsubscribe:
> >> >> > http://www.vtk.org/mailman/listinfo/vtkusers
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Unpaid intern in BillsBasement at noware dot com
> >> >
> >> >
>
>
>
> --
> Unpaid intern in BillsBasement at noware dot com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140304/440de636/attachment.html>
More information about the vtkusers
mailing list