[vtk-developers] zero-copy mixed language support in vtkDataArray

[David Gobbi]

Hi Burlen,

There might be a simpler way to get VTK to free the numpy array,
that won't require any changes to the VTK code at all: just add
the numpy array to the VTK array as an attribute.  Here is some
example code:

== BEGIN EXAMPLE ==

# An example of using numpy arrays in VTK
import vtk
import numpy
import weakref
import gc

# Use a numpy array as a VTK array
z = numpy.arange(0,10,1,float)
a = vtk.vtkDoubleArray()
a.SetVoidArray(z,10,1)

# Make the VTK array track the numpy array
a.array = z

# Create weakrefs to check for deletion
rz = weakref.ref(z)
ra = weakref.ref(a)

# Decref the numpy array, it won't be deleted yet
del z
gc.collect()
if rz() == None:
    print "numpy array is deleted (#1)"

# Decref the VTK array, numpy array will be decref'd
del a
gc.collect()
if rz() == None:
    print "numpy array is deleted (#2)"

== END EXAMPLE ==

This works because the attributes of a VTK-Python object are kept
alive for as long as the corresponding VTK-C++ object exists.

  David

On Sun, Jan 19, 2014 at 12:09 AM, Burlen Loring <burlen.loring at gmail.com> wrote:
> To illustrate the callback in action I wrote an example where data is
> transferred to VTK from numpy ndarrays. I hope this will help. README file
> explains the example. https://github.com/burlen/TestZeroCopyPython
>
>
>>> OK. as fas as VTK is concerned callbackData is intended to be a key
>>> for use by the callback, and VTK does nothing with it beyond passing
>>> it to the callback. I'll add a note about this to clarify.
>>
>> Thanks. The problem with callback context ownership is that if it's a
>> one-time, the callback "owns" the context, but if it's a notification
>> kind of thing, the *notifier* owns the context (since the callback may
>> never be called) and needs an additional function to free the context.
>>
> I'm not following your complaint here. We're concerned about a pointer to
> some array that must be kept alive while VTK uses it but must be released
> when VTK is done. In this situation VTK "no longer needs the data" only ever
> once. and just as VTK always will call free when the pointer is passed in
> through the existing SetArray w/ VTK_DATA_ARRAY_FREE flag. VTK will always
> call the callback, either when the vtkDataArray is resized or when it is
> Delete'd. The callback and callback data are pointers, as far as VTK is
> concerned they're just values, and VTK need not worry about free'ing them.
> Note that this doesn't prevent one from passing an instance of an arbitrary
> class in callbackData. In that case the instance should be deleted in the
> callback.
>
>
>>> Works for me, although I'm not sure NULL callbackData is ever
>>> valid/usefull.
>>
>> As an example, if you have a LocalFree or whatever on Windows, you just
>> need the array parameter, so NULL callbackData is needed.
>
> NULL callbackData would generally not work out. the callback needs non-NULL
> callbackData passed to it in order to identify the memory to free. In your
> example callbackData is the array pointer itself, not NULL. This is not a
> problem.
>
>>   Where array
>> and callbackData might be needed is something like where you need the
>> array, but callbackData is the arena to free it from. Currently, you
>> only get one or the other.
>
> actually I think currently you could get both without issue. don't forget
> callbackData is very flexible, it can be (point to) anything. In the case
> you're describing: write a class containing the arena and the array. pass a
> heap allocated instance of your class as callbackData when you call
> SetArray. When VTK calls the callback, in addition to releasing the array
> data, delete the instance to your class.
>
> Would you be happier with a polymorphic reference counted functor, rather
> than a callback? Initially I thought this would be overkill, but I'd be
> willing to go that way if people are against the callback idea.
>
> Burlen
>
>
> On 01/17/2014 09:19 PM, Ben Boeckel wrote:
>>
>> On Fri, Jan 17, 2014 at 15:21:42 -0800, Burlen Loring wrote:
>>>
>>> Yes, my intention is that for now they'd be used from glue code
>>> rather than exposed directly. In the future the API could be exposed
>>> in the wrapped languages if there were a compelling use case for it.
>>> Until then it's not worth the effort since I doubt VTK wrapping codes
>>> would handle it correctly.
>>
>> That's valid; just making sure :) .
>>
>>> OK. as fas as VTK is concerned callbackData is intended to be a key
>>> for use by the callback, and VTK does nothing with it beyond passing
>>> it to the callback. I'll add a note about this to clarify.
>>
>> Thanks. The problem with callback context ownership is that if it's a
>> one-time, the callback "owns" the context, but if it's a notification
>> kind of thing, the *notifier* owns the context (since the callback may
>> never be called) and needs an additional function to free the context.
>>
>>> Works for me, although I'm not sure NULL callbackData is ever
>>> valid/usefull.
>>
>> As an example, if you have a LocalFree or whatever on Windows, you just
>> need the array parameter, so NULL callbackData is needed. Where array
>> and callbackData might be needed is something like where you need the
>> array, but callbackData is the arena to free it from. Currently, you
>> only get one or the other.
>>
>> --Ben
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.vtk.org/mailman/listinfo/vtk-developers
>