[vtk-developers] VTK python wrapping - returning binary buffer to python?

[Aron Helser]

Thank you for your suggestions and help. I got the ctypes method working in
my code, then discovered that the websocket library I am using requires a
'bytes' object exactly - a bytes-like object won't do. So I am stuck doing
a copy, I think.

Thanks again!
Aron

On Fri, Aug 4, 2017 at 12:22 PM, David Gobbi <david.gobbi at gmail.com> wrote:

> On Fri, Aug 4, 2017 at 10:02 AM, David Gobbi <david.gobbi at gmail.com>
> wrote:
>
>> On Fri, Aug 4, 2017 at 9:08 AM, Aron Helser <aron.helser at kitware.com>
>> wrote:
>>
>>>
>>> Sure thing. And as I trace the code more carefully, I found at least
>>> part of the issue.
>>>
>>> I miss-spoke - the method in question is in ParaView, an override of a
>>> method in VTK:
>>>
>>> ParaView/Web/Core/vtkPVWebApplication.cxx:
>>> vtkUnsignedCharArray* vtkPVWebApplication::StillRender(vtkSMViewProxy*
>>> view, int quality)
>>>
>>> However, that method is wrapped in const char*
>>> vtkPVWebApplication::StillRenderToString(), which does this:
>>>
>>>
>>> return reinterpret_cast<char*>(array->GetPointer(0));
>>>
>>> So that's where the 'char *' to python 'str' conversion happens.
>>>
>>
>> Yes, python will interpret a returned "char *" as a c-style string.  One
>> way to avoid this is to return an std::string, but std::string isn't useful
>> when you're trying to do a zero-copy operation.
>>
>>
>> If I avoid that wrapper and return vtkUnsignedCharArray* directly, can I
>>> get a python 'bytes' object out of it in the Python code without copying?
>>>
>>
>> Yes.  In Python, the vtkUnsignedCharArray (and all other array types)
>> support the Python buffer interface.  That's a feature that Berk added to
>> the wrappers a few years back.
>>
>> So if you were (hypothetically) using numpy, you could use
>> numpy.frombuffer() to interpret the VTK array as a numpy array.  This is a
>> perfect zerocopy operation.
>>
>> The Python builtins, however, are more cautious about zerocopy than VTK
>> or numpy.  So if you were to simply do this, you'd get a copy:
>>
>>   b = bytes(a)  # where "a" is a VTK array object
>>
>> The Python "bytes" object is supposed to own its own memory, it wasn't
>> designed to share.  Even the numpy .tostring() and .tobytes() methods
>> produce copies.
>>
>> However, I suspect that you don't have to convert the array to bytes
>> anyway.  Just pass the VTK array object to whatever method you were
>> planning to pass the 'bytes' object to, and it will probably work fine.
>> Many methods that take 'bytes' will actually take any object that provides
>> a buffer interface, and that includes numpy arrays and VTK arrays.
>>
>
>
> I just remembered something else.  The Python 'ctypes' module is another
> way of doing zerocopy.  Here is some example code:
>
> c_type = ctypes.c_char*n # this means 'char [n]' c-style array
> c_array = c_type.from_buffer(a) # where 'a' is a VTK array
>
> So c_array is now shares memory with the VTK array.  And you can use
> 'c_array.raw' to convert it into 'bytes', but like all the other methods we
> have seen that produce 'bytes', it produces a copy.
>
>  - David
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20170806/057378e1/attachment.html>