[vtk-developers] zero-copy mixed language support in vtkDataArray

[David Gobbi]

I meant vtkCommand, not vtkCallback.  Invoke a FreeEvent and catch it
on the other side with a vtkCommand.

On Sun, Jan 19, 2014 at 4:41 PM, David Gobbi <david.gobbi at gmail.com> wrote:
> Hi Burlen,
>
> If that trick won't work, then maybe the easiest thing is to make the
> vtkDataArray generate a specific event (e.g. create a new event called
> a FreeEvent) that is called when the memory is freed.  That way you
> are taking advantage of the existing vtkCallback support, which is
> already wrapped by all of the wrapper languages.
>
>   David
>
> On Sun, Jan 19, 2014 at 4:22 PM, Burlen Loring <burlen.loring at gmail.com> wrote:
>> thanks, that's a neat trick!
>>
>> it certainly keeps one safe : the numpy object is around while the array is.
>> I don't think that this will do the right thing if the vtk array is resized,
>> or if SetVoidPointer is called again. In both cases, the vtk data array is
>> still alive and well, but the numpy array's ref count should be decremented.
>>
>> a python only solution wont work for me here, because I'm working in the
>> C/C++ "glue layer" sitting in between the python app(s) (which aren't mine)
>> and a legacy VTK based app. VTK objects aren't used in the app's python API
>> at all,  only lists, tuples, and I'm adding support for numpy arrays. Using
>> VTK objects in the API is not an option.
>>
>>
>> On 1/19/2014 8:43 AM, David Gobbi wrote:
>>>
>>> Hi Burlen,
>>>
>>> There might be a simpler way to get VTK to free the numpy array,
>>> that won't require any changes to the VTK code at all: just add
>>> the numpy array to the VTK array as an attribute.  Here is some
>>> example code:
>>>
>>> == BEGIN EXAMPLE ==
>>>
>>> # An example of using numpy arrays in VTK
>>> import vtk
>>> import numpy
>>> import weakref
>>> import gc
>>>
>>> # Use a numpy array as a VTK array
>>> z = numpy.arange(0,10,1,float)
>>> a = vtk.vtkDoubleArray()
>>> a.SetVoidArray(z,10,1)
>>>
>>> # Make the VTK array track the numpy array
>>> a.array = z
>>>
>>> # Create weakrefs to check for deletion
>>> rz = weakref.ref(z)
>>> ra = weakref.ref(a)
>>>
>>> # Decref the numpy array, it won't be deleted yet
>>> del z
>>> gc.collect()
>>> if rz() == None:
>>>      print "numpy array is deleted (#1)"
>>>
>>> # Decref the VTK array, numpy array will be decref'd
>>> del a
>>> gc.collect()
>>> if rz() == None:
>>>      print "numpy array is deleted (#2)"
>>>
>>> == END EXAMPLE ==
>>>
>>> This works because the attributes of a VTK-Python object are kept
>>> alive for as long as the corresponding VTK-C++ object exists.
>>>
>>>    David
>>>
>>> On Sun, Jan 19, 2014 at 12:09 AM, Burlen Loring <burlen.loring at gmail.com>
>>> wrote:
>>>>
>>>> To illustrate the callback in action I wrote an example where data is
>>>> transferred to VTK from numpy ndarrays. I hope this will help. README
>>>> file
>>>> explains the example. https://github.com/burlen/TestZeroCopyPython
>>>>
>>>>
>>>>>> OK. as fas as VTK is concerned callbackData is intended to be a key
>>>>>> for use by the callback, and VTK does nothing with it beyond passing
>>>>>> it to the callback. I'll add a note about this to clarify.
>>>>>
>>>>> Thanks. The problem with callback context ownership is that if it's a
>>>>> one-time, the callback "owns" the context, but if it's a notification
>>>>> kind of thing, the *notifier* owns the context (since the callback may
>>>>> never be called) and needs an additional function to free the context.
>>>>>
>>>> I'm not following your complaint here. We're concerned about a pointer to
>>>> some array that must be kept alive while VTK uses it but must be released
>>>> when VTK is done. In this situation VTK "no longer needs the data" only
>>>> ever
>>>> once. and just as VTK always will call free when the pointer is passed in
>>>> through the existing SetArray w/ VTK_DATA_ARRAY_FREE flag. VTK will
>>>> always
>>>> call the callback, either when the vtkDataArray is resized or when it is
>>>> Delete'd. The callback and callback data are pointers, as far as VTK is
>>>> concerned they're just values, and VTK need not worry about free'ing
>>>> them.
>>>> Note that this doesn't prevent one from passing an instance of an
>>>> arbitrary
>>>> class in callbackData. In that case the instance should be deleted in the
>>>> callback.
>>>>
>>>>
>>>>>> Works for me, although I'm not sure NULL callbackData is ever
>>>>>> valid/usefull.
>>>>>
>>>>> As an example, if you have a LocalFree or whatever on Windows, you just
>>>>> need the array parameter, so NULL callbackData is needed.
>>>>
>>>> NULL callbackData would generally not work out. the callback needs
>>>> non-NULL
>>>> callbackData passed to it in order to identify the memory to free. In
>>>> your
>>>> example callbackData is the array pointer itself, not NULL. This is not a
>>>> problem.
>>>>
>>>>>    Where array
>>>>> and callbackData might be needed is something like where you need the
>>>>> array, but callbackData is the arena to free it from. Currently, you
>>>>> only get one or the other.
>>>>
>>>> actually I think currently you could get both without issue. don't forget
>>>> callbackData is very flexible, it can be (point to) anything. In the case
>>>> you're describing: write a class containing the arena and the array. pass
>>>> a
>>>> heap allocated instance of your class as callbackData when you call
>>>> SetArray. When VTK calls the callback, in addition to releasing the array
>>>> data, delete the instance to your class.
>>>>
>>>> Would you be happier with a polymorphic reference counted functor, rather
>>>> than a callback? Initially I thought this would be overkill, but I'd be
>>>> willing to go that way if people are against the callback idea.
>>>>
>>>> Burlen
>>>>
>>>>
>>>> On 01/17/2014 09:19 PM, Ben Boeckel wrote:
>>>>>
>>>>> On Fri, Jan 17, 2014 at 15:21:42 -0800, Burlen Loring wrote:
>>>>>>
>>>>>> Yes, my intention is that for now they'd be used from glue code
>>>>>> rather than exposed directly. In the future the API could be exposed
>>>>>> in the wrapped languages if there were a compelling use case for it.
>>>>>> Until then it's not worth the effort since I doubt VTK wrapping codes
>>>>>> would handle it correctly.
>>>>>
>>>>> That's valid; just making sure :) .
>>>>>
>>>>>> OK. as fas as VTK is concerned callbackData is intended to be a key
>>>>>> for use by the callback, and VTK does nothing with it beyond passing
>>>>>> it to the callback. I'll add a note about this to clarify.
>>>>>
>>>>> Thanks. The problem with callback context ownership is that if it's a
>>>>> one-time, the callback "owns" the context, but if it's a notification
>>>>> kind of thing, the *notifier* owns the context (since the callback may
>>>>> never be called) and needs an additional function to free the context.
>>>>>
>>>>>> Works for me, although I'm not sure NULL callbackData is ever
>>>>>> valid/usefull.
>>>>>
>>>>> As an example, if you have a LocalFree or whatever on Windows, you just
>>>>> need the array parameter, so NULL callbackData is needed. Where array
>>>>> and callbackData might be needed is something like where you need the
>>>>> array, but callbackData is the arena to free it from. Currently, you
>>>>> only get one or the other.
>>>>>
>>>>> --Ben