[vtkusers] Memory leak vtkEnsightGoldReader in Python

[Bernhard Righolt]

Dear all,

A typical case file, looks as follows (ASCII).

$ head -n 20 case.case
FORMAT
type: ensight gold

GEOMETRY
model:        1     sliceCentre.000.mesh

VARIABLE
vector per node:         1       U       sliceCentre.*****.U

TIME
time set:                      1
number of steps:               11550
filename start number:         1
filename increment:            1
time values:
0.002
0.004
0.006
0.008
0.01
$

The mesh (snippet only), is as follows:

$ head -n 10 sliceCentre.000.mesh
Ensight Geometry File
=====================
node id assign
element id assign
part
         1
sliceCentre.000.mesh
coordinates
    299028
 0.00000e+00
$ sed -n -e 897091,897098p sliceCentre.000.mesh
 0.00000e+00
 0.00000e+00
 0.00000e+00
tria3
    595264
    162106    162714    162799
    162799    162226    162106
    162714    162106    162021


And the corresponding datafile
$ head -n 10 sliceCentre.00001.U
vector
part
         1
coordinates
 5.33425e-04
 2.69515e-03
-1.65164e-03
 5.33425e-04
 1.39069e-04
 2.69515e-03


However, I doubt if something is wrong with my datafiles itself, because I
can easily read them in Paraview for example, the only difference is the
large amount of data-files, e.g (124GB in this case)

Best regards,
Bernhard


On 27 February 2014 22:28, Berk Geveci <berk.geveci at kitware.com> wrote:

> Nothing strikes me as obviously wrong in your script. Can you send me
> your case file? I'll take a look.
>
> -berk
>
> On Thu, Feb 27, 2014 at 3:20 AM, Bernhard Righolt
> <b.w.righolt at tudelft.nl> wrote:
> > Dear all,
> >
> > Currently I am using a Python script to process information stored in the
> > EnSight Gold format. My Python (2.6) scipt uses VTK (5.10.0) to process
> the
> > file, where I used the vtkEnSightGoldReader for reading the data, and
> loop
> > over time steps.
> >
> > In principle this works fine for smaller datasets, however, for large
> > datasets (i.e. 11k files of 12MB), I see the memory usage (recorded via
> top)
> > increasing with time while the process is running. This filling of the
> > memory goes slow, but in some cases problems are inevitable. Apparently,
> I
> > am doing some things wrong in my script with respect to memory
> management.
> >
> > I am using the following script (stripped obviously, but still showing
> this
> > issue)
> >
> > import vtk
> >
> > reader = vtk.vtkEnSightGoldReader()
> >
> > reader.SetCaseFileName("case.case")
> > reader.Update()
> >
> > # Get time values
> > timeset=reader.GetTimeSets()
> > time=timeset.GetItem(0)
> > timesteps=time.GetSize()
> >
> > #reader.ReleaseDataFlagOn()
> >
> > for j in range(timesteps):
> >     curTime=time.GetTuple(j)[0]
> >     print curTime
> >     reader.SetTimeValue(curTime)
> >     reader.Update()
> >
> >     #reader.RemoveAllInputs()
> >
> >
> > I tried a few things as you can see
> > - reader.ReleaseDataFlagOn(), without success
> > - reader.RemoveAllInputs(), did not get it to run
> > - running gc.collect at the end of the loop, following
> > http://vtk.1045678.n5.nabble.com/VTK-memory-management-td5715661.html
> > - Using DeepCopy as suggested here:
> >
> http://vtk.1045678.n5.nabble.com/Memory-leak-in-VTK-Python-module-td1234002.html
> > However, I am already experiencing this without a call to GetOutput
> >
> >
> > My question is, how can I properly unload/replace the data that is
> stored in
> > the memory, instead of using more memory continuously?
> >
> > Best regards,
> > Bernhard
> >
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the VTK FAQ at:
> > http://www.vtk.org/Wiki/VTK_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.vtk.org/mailman/listinfo/vtkusers
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140303/f855e82a/attachment.html>