[vtkusers] segfault on cell picking, and solution

[Robert Dean]

Hello,

For a long time I had an issue where my application would randomly crash
while picking cells. Each time the stack dump was in the depths of  the vtk
code.  What I tracked it down to was that when an actor is first clicked,
vtk allocates an internal cell array to assist in mapping the click to cell
index.  This array is not automatically resized on future clicks.

Therefore when the actor is first clicked an array of N is allocated. If
the array is then updated to contain N*2 cells, clicking cells N through
2N-1 will access outside the cell index array, resulting in the segfault.

As cell picking is a rare event in the application timing, the solution was
to call polydata->DeleteCells() method on each update.

Posting in case someone else runs into the same problem.   I am currently
using vtk 5.10, so I do not know if a newer version addresses this issue.

-Bob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.vtk.org/pipermail/vtkusers/attachments/20130301/71c354c9/attachment.htm>