[vtkusers] Creating a very large VTK array from a numpy array

[David Gobbi]

Hi Jim,

I traced through the VTK code, and the problem occurs when VTK tries
to access the array through the array's buffer interface, specifically when
it calls bf_getreadbuffer() on the array.  The call fails, but numpy doesn't
set an exception to say why.

Unfortunately, I don't have time to delve into the numpy code this week,
so can you submit a bug report at http://www.vtk.org/Bug to make sure
that this issue is not forgotten?

 - David


On Mon, Sep 26, 2011 at 10:26 AM, David Gobbi <david.gobbi at gmail.com> wrote:
> Hi Jim,
>
> I've confirmed this error on my own machine and will start working on
> a fix soon.  That second argument isn't the problem, the "error return
> without exception" also occurs for SetImportVoidPointer(a).  When I
> have a fix, I'll report back.
>
> You can save memory by changing the code a bit:
>
> # create uint8 array directly, rather than as double with conversion to uint8
> a = numpy.zeros(2**32-1, numpy.uint8)
> # directly use numpy array in VTK, avoid "tostring()"
> im.SetImportVoidPointer(a)
>
>  - David
>
>
> On Mon, Sep 26, 2011 at 8:25 AM, Jim Geach <jimgeach at physics.mcgill.ca> wrote:
>>
>> Hi List,
>>
>> I'm rendering very large grids that require use of VTK's USE_64BIT_IDS. Following a previous post where I was having trouble with this, I upgraded to version 5.8 which fixed a bug that was not allowing ImageData to use the 64bit indexing (and this flag was set to "ON" in the CMake, and the machine is more than capable).
>>
>> Now I've got a new problem: I'm working from Python, and creating VTK arrays from numpy arrays, but I still run into problems when the number of elements gets too large. As an example of what I'm talking about:
>>
>> from numpy import zeros
>> import vtk
>>
>> a = zeros(2**32 - 1).astype(uint8)              #to create a massive array
>> im = vtk.vtkImageImport()                       #believe this is the standard method to convert to vtk
>> im.CopyImportVoidPointer(a.tostring(),a.size)   #do conversion
>>
>> throws the following error:
>>
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>> SystemError: error return without exception set
>>
>> I'm guessing the type of the second argument of CopyImportVoidPointer is the key here, but trying a few different castings I can't get it to work.
>>
>> So, I guess my question boils down to: how do I convert extremely large numpy arrays into vtk arrays using the standard methods? (Note that in practice, my numpy arrays will consist of unsigned chars for volume rendering).
>>
>> Thanks, Jim