[vtkusers] relatively obscure picking bug

[Michael Halle]

I've been looking at the vtk cell picking code today, and I think I've
found an bug/oversight of the current implementation.  In summary,
picking composite cells like triangle strips may not produce the
correct result when several parts of the cell intersect the view ray
(more details below).  Not sure if it should be fixed or just
documented.

Inside vtkCellPicker (and vtkPicker, its superclass), vtk looks
through all of the cells of all of the props and intersects them with
the view ray.  The picker then returns the frontmost of the results.
Sounds great.

To do the intersection, the cell picker calls the IntersectWithLine
method of each cell.  For primitive cells, this operation is
straightforward and accurate.  However, for composite cells, the
intersection method gets called for each contained cell, and the
*first one* (not the closest one) that intersects the ray is returned.
If you have a twisty triangle strip seen in relief, your viewing ray
might intersect the ray twice or more.  There's no telling which
intersection you'll get.  vtkTriangleStrip is probably the most
widely-used of the affected classes, but the other vtkPoly* cell types
seem vulnerable as well.

One solution to the problem would be to just fix the affected classes
to do a sort, returning the frontmost intersection.  On the other
hand, what I was actually trying to do when I started this epic
journey into the vtk source code was to write a new cell picker class
that returned *all* intersections (with points, cell ids, and sub
ids), not just the first one.  It would be nice if there was a way
to get all that data out of a cell -- otherwise, what I'm trying
to do is difficult without excessive muckage.  

How to approach this problem?  Thanks.


Michael Halle
mhalle at bwh.harvard.edu