[vtk-developers] VTK python wrapping - returning binary buffer to python?

[David Gobbi]

On Fri, Aug 4, 2017 at 10:02 AM, David Gobbi <david.gobbi at gmail.com> wrote:

> On Fri, Aug 4, 2017 at 9:08 AM, Aron Helser <aron.helser at kitware.com>
> wrote:
>
>>
>> Sure thing. And as I trace the code more carefully, I found at least part
>> of the issue.
>>
>> I miss-spoke - the method in question is in ParaView, an override of a
>> method in VTK:
>>
>> ParaView/Web/Core/vtkPVWebApplication.cxx:
>> vtkUnsignedCharArray* vtkPVWebApplication::StillRender(vtkSMViewProxy*
>> view, int quality)
>>
>> However, that method is wrapped in const char*
>> vtkPVWebApplication::StillRenderToString(), which does this:
>>
>>
>> return reinterpret_cast<char*>(array->GetPointer(0));
>>
>> So that's where the 'char *' to python 'str' conversion happens.
>>
>
> Yes, python will interpret a returned "char *" as a c-style string.  One
> way to avoid this is to return an std::string, but std::string isn't useful
> when you're trying to do a zero-copy operation.
>
>
> If I avoid that wrapper and return vtkUnsignedCharArray* directly, can I
>> get a python 'bytes' object out of it in the Python code without copying?
>>
>
> Yes.  In Python, the vtkUnsignedCharArray (and all other array types)
> support the Python buffer interface.  That's a feature that Berk added to
> the wrappers a few years back.
>
> So if you were (hypothetically) using numpy, you could use
> numpy.frombuffer() to interpret the VTK array as a numpy array.  This is a
> perfect zerocopy operation.
>
> The Python builtins, however, are more cautious about zerocopy than VTK or
> numpy.  So if you were to simply do this, you'd get a copy:
>
>   b = bytes(a)  # where "a" is a VTK array object
>
> The Python "bytes" object is supposed to own its own memory, it wasn't
> designed to share.  Even the numpy .tostring() and .tobytes() methods
> produce copies.
>
> However, I suspect that you don't have to convert the array to bytes
> anyway.  Just pass the VTK array object to whatever method you were
> planning to pass the 'bytes' object to, and it will probably work fine.
> Many methods that take 'bytes' will actually take any object that provides
> a buffer interface, and that includes numpy arrays and VTK arrays.
>


I just remembered something else.  The Python 'ctypes' module is another
way of doing zerocopy.  Here is some example code:

c_type = ctypes.c_char*n # this means 'char [n]' c-style array
c_array = c_type.from_buffer(a) # where 'a' is a VTK array

So c_array is now shares memory with the VTK array.  And you can use
'c_array.raw' to convert it into 'bytes', but like all the other methods we
have seen that produce 'bytes', it produces a copy.

 - David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20170804/9bf729a3/attachment.html>