[Insight-users] itk image from itk.PyBuffer, problem when used as function parameter

Tue Jun 23 18:00:30 EDT 2009

Hi Rick,

I reproduced the bug you had, and put a fix in cvs.

http://public.kitware.com/cgi-bin/viewcvs.cgi/Wrapping/WrapITK/ExternalProjects/PyBuffer/itkPyBuffer.txx?root=Insight&r1=1.2&r2=1.3

That's a quite difficult bug, and I'm not sure the fix will work for  
you, as I'm not sure what it the cause. It seems to be caused by the  
importer being destroyed after the image is used in the writer,  
leading to quite strange behavior.

Can you please test with the fix and report any failure, or success?

Thanks for the report, and for your intensive testing of PyBuffer.

Gaëtan

Le 30 mai 09 à 20:20, Rick Giuly a écrit :

>
> Hi Gaetan,
>
> I'm now running into another problem. It seems that if the itk image  
> is passed as a parameter to a function, it becomes damaged in some  
> way. When I run this code, the itk.write operation works when not  
> inside of a function-- but fails when it is inside of a function.
>
> -Rick
>
> ----
>
> import itk, sys, gc
> itk.auto_progress()
> import numpy
>
> # image size is 1 GB
> # IT = itk.Image.UC3
> # img = IT.New(Regions=1000)
>
> # image size is 1 GB
> IT = itk.Image.F3
> #img = IT.New(Regions=[1000, 1000, 250])
> img = IT.New(Regions=[1000, 100, 250])
>
> img.Allocate()
> img.FillBuffer(0)
> itk.write(img, "/tmp/result.nrrd")
>
> print img
>
> # exercise buffer conversion from itk to numpy
> for i in range(1000):
> buf = itk.PyBuffer[IT].GetArrayFromImage(img)
> #buf = zeros((1000, 1000, 250))
> del buf
>
> # exercise buffer conversion from numpy to itk
>
> count = 0
>
> def callback():
> global count
> sys.stderr.write(str(count)+"\t")
> if count%10 == 0:
>   sys.stderr.write("\n")
> sys.stderr.flush()
> count += 1
> com = itk.PyCommand.New()
> com.SetCommandCallable( callback )
>
> buf = itk.PyBuffer[IT].GetArrayFromImage(img)
> #buf = numpy.zeros((100, 100, 250))
> buf[1, 2, 3] = 20
> buf[:, :, :] = numpy.ones((250, 100, 1000))
> #buf[:, :, :] = 1
>
> def functionTest(volume):
>    itk.write(volume, "/tmp/temp.nrrd")
>
> for i in range(100000):
> print i
> print buf[1, 2, 3]
> print buf[1, 2, 4]
> bi = itk.PyBuffer[IT].GetImageFromArray(buf)
> itk.write(bi, "/tmp/result2.nrrd")
> functionTest(bi)
> #bi.AddObserver( itk.DeleteEvent(), com )
> ImageType = itk.Image[itk.F, 3]
> converter = itk.PyBuffer[ImageType]
> #buf2 = itk.PyBuffer[IT].GetArrayFromImage(bi)
> buf2 = converter.GetArrayFromImage(bi)
> del bi
>
> # force garbage collection
> gc.collect()
>
> sys.stderr.write(str(count)+"\n")
>
> sys.exit(abs(1000-count))
>
> Gaëtan Lehmann wrote:
>> My test also work on the stable version of wrapitk (with a few  
>> changes in command manipulation).
>> I tried with the test you provided, and this time I get an error:
>>   RuntimeError: Contiguous array couldn't be created from input  
>> python object
>> This error is sent from PyBuffer when it can't convert the input  
>> python object to the expected type. Maybe the array returned by  
>> numpy.ones() is not a contigus array?
>> Gaëtan
>> Le 22 mai 09 à 15:56, Gaëtan Lehmann a écrit :
>>>
>>> Hi Rick,
>>>
>>> Yes, I looked at it in wrapitk *unstable* - it may not be the one  
>>> you're using.
>>> I can't reproduce the problem you have. Here is the test I've run:
>>>
>>> #!/usr/bin/env python
>>>
>>> import itk, sys, gc
>>> itk.auto_progress()
>>>
>>> # image size is 1 GB
>>> # IT = itk.Image.UC3
>>> # img = IT.New(Regions=1000)
>>>
>>> # image size is 1 GB
>>> IT = itk.Image.F3
>>> img = IT.New(Regions=[1000, 1000, 250])
>>>
>>> img.Allocate()
>>> img.FillBuffer(0)
>>>
>>> print img
>>>
>>> # exercise buffer conversion from itk to numpy
>>> for i in range(1000):
>>> buf = itk.PyBuffer[IT].GetArrayFromImage(img)
>>> del buf
>>>
>>> # exercise buffer conversion from numpy to itk
>>>
>>> count = 0
>>>
>>> def callback():
>>> global count
>>> sys.stderr.write(str(count)+"\t")
>>> if count%10 == 0:
>>>   sys.stderr.write("\n")
>>> sys.stderr.flush()
>>> count += 1
>>> com = itk.PyCommand.New()
>>> com.SetCommandCallable( callback )
>>>
>>> buf = itk.PyBuffer[IT].GetArrayFromImage(img)
>>>
>>> for i in range(1000):
>>> bi = itk.PyBuffer[IT].GetImageFromArray(buf)
>>> bi.AddObserver( itk.DeleteEvent(), com )
>>> del bi
>>>
>>> # force garbage collection
>>> gc.collect()
>>>
>>> sys.stderr.write(str(count)+"\n")
>>>
>>> sys.exit(abs(1000-count))
>>>
>>>
>>>
>>>
>>> I'm now looking at wrapitk stable. I'll let you know what I'll  
>>> found.
>>>
>>> Gaëtan
>>>
>>>
>>> Le 22 mai 09 à 06:16, Rick Giuly a écrit :
>>>
>>>> Hi Gaëtan,
>>>>
>>>> Have you had a chance to look at this?
>>>>
>>>> thanks
>>>>
>>>> -rick
>>>>
>>>> Gaëtan Lehmann wrote:
>>>>> Le 13 mai 09 à 10:35, Rick Giuly a écrit :
>>>>>>
>>>>>> I've now tested invoking garbage collection with gc.collect()  
>>>>>> at every iteration and that didn't change the problem. So, my  
>>>>>> best guess would be that the image is not deleted when the  
>>>>>> python object is destroyed. Maybe a bug?
>>>>> Maybe - can you file a bug on ITK's bug tracker and assign it to  
>>>>> me?
>>>>> I'll try to find a bit of time to investigate this in the next  
>>>>> days.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Gaëtan Lehmann wrote:
>>>>>>> Hi Rick,
>>>>>>> GetImageFromArray() returns a smart pointer to an image, so  
>>>>>>> the image should be deallocated once the python object is  
>>>>>>> destroyed.
>>>>>>> Perhaps the problem is there: the garbage collection my not  
>>>>>>> run fast enough.
>>>>>>> The right way to go may be to implement PyBuffer as a filter  
>>>>>>> which reuse the output image, as all the filters in itk.
>>>>>>> For the Delete() method: you shouldn't use it in python (and I  
>>>>>>> can't see any case for use it but the internal memory  
>>>>>>> management in c++). In the latest version of wrapitk (the one  
>>>>>>> hosted at googlecode), Delete() and the other methods related  
>>>>>>> to smart pointers are hidden.
>>>>>>> Regards,
>>>>>>> Gaëtan
>>>>>>> Le 11 mai 09 à 05:43, Rick Giuly a écrit :
>>>>>>>>
>>>>>>>> Hello All,
>>>>>>>>
>>>>>>>> It seems that converter.GetImageFromArray(inputNumpyVolume)  
>>>>>>>> allocates memory in some way and never releases it. When I  
>>>>>>>> tried using Delete() the program actually crashed silently  
>>>>>>>> after one iteration.
>>>>>>>>
>>>>>>>> Test code is below. (I'm running this on ubuntu, and the itk  
>>>>>>>> package is from Paul Novo's site.)
>>>>>>>>
>>>>>>>> Is there some way to release memory?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> import itk
>>>>>>>> import numpy
>>>>>>>>
>>>>>>>> for i in range(10000):
>>>>>>>>
>>>>>>>> print i
>>>>>>>>
>>>>>>>> ImageType = itk.Image[itk.F, 3]
>>>>>>>> converter = itk.PyBuffer[ImageType]
>>>>>>>>
>>>>>>>> inputNumpyVolume = numpy.ones((100, 100, 200))
>>>>>>>> inputVolume = converter.GetImageFromArray(inputNumpyVolume)
>>>>>>>> #inputVolume.Delete()
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----------
>>>>>>>> Thanks,
>>>>>>>> --Rick
>>>>>>>> _____________________________________
>>>>>>>> Powered by www.kitware.com
>>>>>>>>
>>>>>>>> Visit other Kitware open-source projects at
>>>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>>>
>>>>>>>> Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ
>>>>>>>>
>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>> http://www.itk.org/mailman/listinfo/insight-users
>>>>>>
>>>>
>>>
>>> -- 
>>> Gaëtan Lehmann
>>> Biologie du Développement et de la Reproduction
>>> INRA de Jouy-en-Josas (France)
>>> tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
>>> http://voxel.jouy.inra.fr  http://www.mandriva.org
>>> http://www.itk.org  http://www.clavier-dvorak.org
>>>
>

-- 
Gaëtan Lehmann
Biologie du Développement et de la Reproduction
INRA de Jouy-en-Josas (France)
tel: +33 1 34 65 29 66    fax: 01 34 65 29 09
http://voxel.jouy.inra.fr  http://www.itk.org
http://www.mandriva.org  http://www.bepo.fr

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: Ceci est une signature ?lectronique PGP
URL: <http://www.itk.org/pipermail/insight-users/attachments/20090624/fbcfae65/attachment.pgp>