[vtk-developers] VTK python wrapping - returning binary buffer to python?

[Aron Helser]

One more wrinkle, of course!
This code you suggested works fine in Python3:

  b = bytes(a)  # where "a" is a VTK array object


Unfortunately, it doesn't work in Python 2.7 - it's returning the
equivalent of str(a), which is info about the object:
'vtkUnsignedCharArray (00000000124BC8A0)\n  Debug: Off\n  Modified Time:
274789\n ....'

So I'm going to switch to this:
b = memoryview(a).tobytes()

which will work in Python 2.7 and above, and should result in just one copy.
Regards,
Aron


On Sun, Aug 6, 2017 at 12:36 PM, Aron Helser <aron.helser at kitware.com>
wrote:

> Thank you for your suggestions and help. I got the ctypes method working
> in my code, then discovered that the websocket library I am using requires
> a 'bytes' object exactly - a bytes-like object won't do. So I am stuck
> doing a copy, I think.
>
> Thanks again!
> Aron
>
> On Fri, Aug 4, 2017 at 12:22 PM, David Gobbi <david.gobbi at gmail.com>
> wrote:
>
>> On Fri, Aug 4, 2017 at 10:02 AM, David Gobbi <david.gobbi at gmail.com>
>> wrote:
>>
>>> On Fri, Aug 4, 2017 at 9:08 AM, Aron Helser <aron.helser at kitware.com>
>>> wrote:
>>>
>>>>
>>>> Sure thing. And as I trace the code more carefully, I found at least
>>>> part of the issue.
>>>>
>>>> I miss-spoke - the method in question is in ParaView, an override of a
>>>> method in VTK:
>>>>
>>>> ParaView/Web/Core/vtkPVWebApplication.cxx:
>>>> vtkUnsignedCharArray* vtkPVWebApplication::StillRender(vtkSMViewProxy*
>>>> view, int quality)
>>>>
>>>> However, that method is wrapped in const char*
>>>> vtkPVWebApplication::StillRenderToString(), which does this:
>>>>
>>>>
>>>> return reinterpret_cast<char*>(array->GetPointer(0));
>>>>
>>>> So that's where the 'char *' to python 'str' conversion happens.
>>>>
>>>
>>> Yes, python will interpret a returned "char *" as a c-style string.  One
>>> way to avoid this is to return an std::string, but std::string isn't useful
>>> when you're trying to do a zero-copy operation.
>>>
>>>
>>> If I avoid that wrapper and return vtkUnsignedCharArray* directly, can I
>>>> get a python 'bytes' object out of it in the Python code without copying?
>>>>
>>>
>>> Yes.  In Python, the vtkUnsignedCharArray (and all other array types)
>>> support the Python buffer interface.  That's a feature that Berk added to
>>> the wrappers a few years back.
>>>
>>> So if you were (hypothetically) using numpy, you could use
>>> numpy.frombuffer() to interpret the VTK array as a numpy array.  This is a
>>> perfect zerocopy operation.
>>>
>>> The Python builtins, however, are more cautious about zerocopy than VTK
>>> or numpy.  So if you were to simply do this, you'd get a copy:
>>>
>>>   b = bytes(a)  # where "a" is a VTK array object
>>>
>>> The Python "bytes" object is supposed to own its own memory, it wasn't
>>> designed to share.  Even the numpy .tostring() and .tobytes() methods
>>> produce copies.
>>>
>>> However, I suspect that you don't have to convert the array to bytes
>>> anyway.  Just pass the VTK array object to whatever method you were
>>> planning to pass the 'bytes' object to, and it will probably work fine.
>>> Many methods that take 'bytes' will actually take any object that provides
>>> a buffer interface, and that includes numpy arrays and VTK arrays.
>>>
>>
>>
>> I just remembered something else.  The Python 'ctypes' module is another
>> way of doing zerocopy.  Here is some example code:
>>
>> c_type = ctypes.c_char*n # this means 'char [n]' c-style array
>> c_array = c_type.from_buffer(a) # where 'a' is a VTK array
>>
>> So c_array is now shares memory with the VTK array.  And you can use
>> 'c_array.raw' to convert it into 'bytes', but like all the other methods we
>> have seen that produce 'bytes', it produces a copy.
>>
>>  - David
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20170807/db48af51/attachment.html>