[vtkusers] Memory leak vtkEnsightGoldReader in Python
Bill Lorensen
bill.lorensen at gmail.com
Tue Mar 4 15:27:15 EST 2014
Attached is my C++ program and CMakeLists.txt file. But I don't think
this is a python issue...
On Tue, Mar 4, 2014 at 3:10 PM, Bernhard Righolt <b.w.righolt at tudelft.nl> wrote:
> Dear Bill,
>
> I am a C++ speaker, so I don't mind to test your code on my current large
> dataset. Can you maybe share your code, such that I can test it? I've never
> used VTK within C++, if it solves the issues I am experiencing with Python,
> I am very willing to switch. I hope it is not too complicated to link the
> libraries correctly.
>
> Bernhard
>
>
>
> On 4 March 2014 21:05, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>
>> Bernhard,
>>
>> Thanks for your patience. Since I'm not a python person, I converted
>> your program to C++. I ran valgrind and found no memory leaks.
>>
>> Perhaps the small dataset you provided does not exercise the code that
>> is leaking.
>>
>> Can you possible place a dataset that you know has leaksat some accessible
>> site?
>>
>> I will say, that I don't think any of the current ensight tests access
>> multiple time steps. Your data and test script will be a good starting
>> point for such a test.
>>
>> Bill
>>
>> On Tue, Mar 4, 2014 at 12:08 AM, Bernhard Righolt
>> <b.w.righolt at tudelft.nl> wrote:
>> > Therr is a python script inside the archive which is the same script
>> > that I
>> > used on the large case.
>> >
>> > Bernhard
>> >
>> >
>> > On Mar 4, 2014 12:22 AM, "Bill Lorensen" <bill.lorensen at gmail.com>
>> > wrote:
>> >>
>> >> This is great. Can you send a complete python script that illustrates
>> >> the
>> >> problem.?
>> >>
>> >> On Mar 3, 2014 2:47 PM, "Bernhard Righolt" <b.w.righolt at tudelft.nl>
>> >> wrote:
>> >> >
>> >> > Hi Bill,
>> >> >
>> >> > I added a very small sample data set. I am not sure if the memory
>> >> > problem I am encountering with my big data-set is measurable for this
>> >> > one.
>> >> >
>> >> > Anyhow, I would like to repeat that I am using Python 2.6 and VTK
>> >> > 5.10,
>> >> >
>> >> > Best regards,
>> >> > Bernhard
>> >> >
>> >> >
>> >> > On 3 March 2014 19:25, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>> >> >>
>> >> >> The ensight readers could use more testing. Can you provide a
>> >> >> "small"
>> >> >> dataset and program that cause memory leaks?
>> >> >>
>> >> >> Bill
>> >> >>
>> >> >>
>> >> >> On Mon, Mar 3, 2014 at 10:38 AM, Bernhard Righolt
>> >> >> <b.w.righolt at tudelft.nl> wrote:
>> >> >> > Dear all,
>> >> >> >
>> >> >> > A typical case file, looks as follows (ASCII).
>> >> >> >
>> >> >> > $ head -n 20 case.case
>> >> >> > FORMAT
>> >> >> > type: ensight gold
>> >> >> >
>> >> >> > GEOMETRY
>> >> >> > model: 1 sliceCentre.000.mesh
>> >> >> >
>> >> >> > VARIABLE
>> >> >> > vector per node: 1 U sliceCentre.*****.U
>> >> >> >
>> >> >> > TIME
>> >> >> > time set: 1
>> >> >> > number of steps: 11550
>> >> >> > filename start number: 1
>> >> >> > filename increment: 1
>> >> >> > time values:
>> >> >> > 0.002
>> >> >> > 0.004
>> >> >> > 0.006
>> >> >> > 0.008
>> >> >> > 0.01
>> >> >> > $
>> >> >> >
>> >> >> > The mesh (snippet only), is as follows:
>> >> >> >
>> >> >> > $ head -n 10 sliceCentre.000.mesh
>> >> >> > Ensight Geometry File
>> >> >> > =====================
>> >> >> > node id assign
>> >> >> > element id assign
>> >> >> > part
>> >> >> > 1
>> >> >> > sliceCentre.000.mesh
>> >> >> > coordinates
>> >> >> > 299028
>> >> >> > 0.00000e+00
>> >> >> > $ sed -n -e 897091,897098p sliceCentre.000.mesh
>> >> >> > 0.00000e+00
>> >> >> > 0.00000e+00
>> >> >> > 0.00000e+00
>> >> >> > tria3
>> >> >> > 595264
>> >> >> > 162106 162714 162799
>> >> >> > 162799 162226 162106
>> >> >> > 162714 162106 162021
>> >> >> >
>> >> >> >
>> >> >> > And the corresponding datafile
>> >> >> > $ head -n 10 sliceCentre.00001.U
>> >> >> > vector
>> >> >> > part
>> >> >> > 1
>> >> >> > coordinates
>> >> >> > 5.33425e-04
>> >> >> > 2.69515e-03
>> >> >> > -1.65164e-03
>> >> >> > 5.33425e-04
>> >> >> > 1.39069e-04
>> >> >> > 2.69515e-03
>> >> >> >
>> >> >> >
>> >> >> > However, I doubt if something is wrong with my datafiles itself,
>> >> >> > because I
>> >> >> > can easily read them in Paraview for example, the only difference
>> >> >> > is
>> >> >> > the
>> >> >> > large amount of data-files, e.g (124GB in this case)
>> >> >> >
>> >> >> > Best regards,
>> >> >> > Bernhard
>> >> >> >
>> >> >> >
>> >> >> > On 27 February 2014 22:28, Berk Geveci <berk.geveci at kitware.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Nothing strikes me as obviously wrong in your script. Can you
>> >> >> >> send
>> >> >> >> me
>> >> >> >> your case file? I'll take a look.
>> >> >> >>
>> >> >> >> -berk
>> >> >> >>
>> >> >> >> On Thu, Feb 27, 2014 at 3:20 AM, Bernhard Righolt
>> >> >> >> <b.w.righolt at tudelft.nl> wrote:
>> >> >> >> > Dear all,
>> >> >> >> >
>> >> >> >> > Currently I am using a Python script to process information
>> >> >> >> > stored
>> >> >> >> > in
>> >> >> >> > the
>> >> >> >> > EnSight Gold format. My Python (2.6) scipt uses VTK (5.10.0) to
>> >> >> >> > process
>> >> >> >> > the
>> >> >> >> > file, where I used the vtkEnSightGoldReader for reading the
>> >> >> >> > data,
>> >> >> >> > and
>> >> >> >> > loop
>> >> >> >> > over time steps.
>> >> >> >> >
>> >> >> >> > In principle this works fine for smaller datasets, however, for
>> >> >> >> > large
>> >> >> >> > datasets (i.e. 11k files of 12MB), I see the memory usage
>> >> >> >> > (recorded via
>> >> >> >> > top)
>> >> >> >> > increasing with time while the process is running. This filling
>> >> >> >> > of
>> >> >> >> > the
>> >> >> >> > memory goes slow, but in some cases problems are inevitable.
>> >> >> >> > Apparently,
>> >> >> >> > I
>> >> >> >> > am doing some things wrong in my script with respect to memory
>> >> >> >> > management.
>> >> >> >> >
>> >> >> >> > I am using the following script (stripped obviously, but still
>> >> >> >> > showing
>> >> >> >> > this
>> >> >> >> > issue)
>> >> >> >> >
>> >> >> >> > import vtk
>> >> >> >> >
>> >> >> >> > reader = vtk.vtkEnSightGoldReader()
>> >> >> >> >
>> >> >> >> > reader.SetCaseFileName("case.case")
>> >> >> >> > reader.Update()
>> >> >> >> >
>> >> >> >> > # Get time values
>> >> >> >> > timeset=reader.GetTimeSets()
>> >> >> >> > time=timeset.GetItem(0)
>> >> >> >> > timesteps=time.GetSize()
>> >> >> >> >
>> >> >> >> > #reader.ReleaseDataFlagOn()
>> >> >> >> >
>> >> >> >> > for j in range(timesteps):
>> >> >> >> > curTime=time.GetTuple(j)[0]
>> >> >> >> > print curTime
>> >> >> >> > reader.SetTimeValue(curTime)
>> >> >> >> > reader.Update()
>> >> >> >> >
>> >> >> >> > #reader.RemoveAllInputs()
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > I tried a few things as you can see
>> >> >> >> > - reader.ReleaseDataFlagOn(), without success
>> >> >> >> > - reader.RemoveAllInputs(), did not get it to run
>> >> >> >> > - running gc.collect at the end of the loop, following
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > http://vtk.1045678.n5.nabble.com/VTK-memory-management-td5715661.html
>> >> >> >> > - Using DeepCopy as suggested here:
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > http://vtk.1045678.n5.nabble.com/Memory-leak-in-VTK-Python-module-td1234002.html
>> >> >> >> > However, I am already experiencing this without a call to
>> >> >> >> > GetOutput
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > My question is, how can I properly unload/replace the data that
>> >> >> >> > is
>> >> >> >> > stored in
>> >> >> >> > the memory, instead of using more memory continuously?
>> >> >> >> >
>> >> >> >> > Best regards,
>> >> >> >> > Bernhard
>> >> >> >> >
>> >> >> >> > _______________________________________________
>> >> >> >> > Powered by www.kitware.com
>> >> >> >> >
>> >> >> >> > Visit other Kitware open-source projects at
>> >> >> >> > http://www.kitware.com/opensource/opensource.html
>> >> >> >> >
>> >> >> >> > Please keep messages on-topic and check the VTK FAQ at:
>> >> >> >> > http://www.vtk.org/Wiki/VTK_FAQ
>> >> >> >> >
>> >> >> >> > Follow this link to subscribe/unsubscribe:
>> >> >> >> > http://www.vtk.org/mailman/listinfo/vtkusers
>> >> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Powered by www.kitware.com
>> >> >> >
>> >> >> > Visit other Kitware open-source projects at
>> >> >> > http://www.kitware.com/opensource/opensource.html
>> >> >> >
>> >> >> > Please keep messages on-topic and check the VTK FAQ at:
>> >> >> > http://www.vtk.org/Wiki/VTK_FAQ
>> >> >> >
>> >> >> > Follow this link to subscribe/unsubscribe:
>> >> >> > http://www.vtk.org/mailman/listinfo/vtkusers
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Unpaid intern in BillsBasement at noware dot com
>> >> >
>> >> >
>>
>>
>>
>> --
>> Unpaid intern in BillsBasement at noware dot com
>
>
--
Unpaid intern in BillsBasement at noware dot com
-------------- next part --------------
cmake_minimum_required(VERSION 2.8)
PROJECT(ReadEnsight)
find_package(VTK REQUIRED)
include(${VTK_USE_FILE})
add_executable(readEnsight MACOSX_BUNDLE readEnsight)
target_link_libraries(readEnsight ${VTK_LIBRARIES})
-------------- next part --------------
A non-text attachment was scrubbed...
Name: readEnsight.cxx
Type: text/x-c++src
Size: 843 bytes
Desc: not available
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20140304/e0a6bc8f/attachment.cxx>
More information about the vtkusers
mailing list