[vtk-developers] More zero-copy array support for Python

[David Gobbi]

It's a feature that was added in Oct 2010, so it probably pre-dates the
dataset_adapter.



On Fri, Jun 16, 2017 at 1:53 PM, Berk Geveci <berk.geveci at kitware.com>
wrote:

> Interesting. Is this something relatively recent? I know that this didn't
> work in the past.
>
> On Fri, Jun 16, 2017 at 3:12 PM, David Gobbi <david.gobbi at gmail.com>
> wrote:
>
>> Hi Berk,
>>
>> That shouldn't be the case.  When a vtk-python object destructs, its
>> attributes aren't deleted until a future GC cycle determines that the
>> vtk-c++ object has destructed.  E.g.:
>>
>> c = numpy.array([1,2,3], 'f')
>> a = vtk.vtkFloatArray()
>> a.SetArray(c, c.size, True)
>> a.array = c
>>
>> pd = vtk.vtkPointData()
>> pd.AddArray(a)
>> del a
>> del c
>>
>> a = pd.GetArray(0)
>> a.array
>> -> array([ 1.,  2.,  3.], dtype=float32)
>>
>> This behavior is tested in Common/Core/Testing/Python/TestGhost.py.  But
>> I notice that numpy_to_vtk() doesn't use this trick...
>>
>> There could also be other isues that I'm not aware of, so I can look at
>> this in more depth.
>>
>>  - David
>>
>>
>>
>> On Fri, Jun 16, 2017 at 11:18 AM, Berk Geveci <berk.geveci at kitware.com>
>> wrote:
>>
>>> This is fantastic David!!!
>>>
>>> One word of caution to people using numpy array -> VTK array route:
>>>
>>> c = numpy.array([1,2,3], 'f')
>>> a = vtk.vtkFloatArray()
>>> a.SetArray(c, c.size, True)
>>> a.array = c  # keep reference to "c"
>>>
>>> This is safe only as long as you keep the a object around. If you do
>>> something like this, you are in trouble:
>>>
>>> pd = vtk.vtkPointData()
>>> pd.AddArray(a)
>>> del a
>>>
>>> In this case, the Python object goes away and therefore releases the
>>> reference to c. In the dataset_adapter module, I implemented a way around
>>> this issue. I suggest taking a look at that which is in
>>> the numpyTovtkDataArray() method.
>>>
>>> Best,
>>> -berk
>>>
>>>
>>>
>>> On Fri, Jun 16, 2017 at 8:37 AM, David Gobbi <david.gobbi at gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Yesterday I merged the wrapping of the vtkSOADataArrays, and also added
>>>> a new wrapper hint to assist with VTK's zero-copy methods.
>>>>
>>>> Zero-copy is when an array uses an existing block of memory.  In the
>>>> Python wrappers, this generally means making a VTK array from a numpy array
>>>> or vice-versa.  Or making a VTK image from a PIL image.  Or using memmap to
>>>> share a VTK data set among multiple processes.
>>>>
>>>> The new wrapper hint VTK_ZEROCOPY can be applied to method parameters
>>>> that are used for zero-copy operations:
>>>>
>>>>   void vtkFloatArray::SetArray(VTK_ZEROCOPY float *ptr, vtkIdType n,
>>>> int save);
>>>>
>>>> This hint allows the Python wrappers to do the zero-copy magic with
>>>> SetArray:
>>>>
>>>>   import numpy
>>>>   import vtk
>>>>   c = numpy.array([1,2,3], 'f')
>>>>   a = vtk.vtkFloatArray()
>>>>   a.SetArray(c, c.size, True)
>>>>   a.array = c  # keep reference to "c"
>>>>
>>>> Astute VTKers will note that this was already possible with
>>>> SetVoidArray(), as used by the vtk.numpy_support module, but SetArray()
>>>> adds type checking.
>>>>
>>>> Also, SetArray() works with the newly-wrapped vtkSOADataArrays, so
>>>> perhaps in the future numpy_support.numpy_to_vtk() can be expanded to
>>>> support SOA arrays:
>>>>
>>>>   c0 = numpy.array([1,2,3], 'f')
>>>>   c1 = numpy.array([4,5,6], 'f')
>>>>   a = vtk.vtkSOADataArrayTemplate[c0.dtype]()
>>>>   a.SetNumberOfComponents(2)
>>>>   a.SetArray(0, c0, c0.size, True, True)
>>>>   a.SetArray(1, c1, c1.size, True, True)
>>>>
>>>> In the future this could be done with  a =
>>>> vtk.numpy_support.numpy_to_vtk( (c0,c1) ).
>>>>
>>>> I haven't yet implemented the mechanism to go in the other direction,
>>>> i.e. to create a numpy array from a VTK SOA array.
>>>>
>>>> Also: vtkSOADataArrayTemplate is now wrapped, but
>>>> vtkAOSDataArrayTemplate is not.  Currently the wrappers think that
>>>> vtkDataArray is the immediate superclass of vtkFloatArray etc.  This could
>>>> be changed, if there are use cases for directly instantiating
>>>> vtkAOSDataArrayTemplate.
>>>>
>>>> Cheers,
>>>>  - David
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20170616/ede2f4ca/attachment-0001.html>