[vtk-developers] vtkOBBTree sqrt domain err

[David Gobbi]

Hi Dean,

I took a look at the LineIntersectsNode() code, trying to figure out why a
range error would occur:

    rangeAmin = vtkMath::Dot( pA->Corner, pA->Axes[ii] );
    rangeAmax = rangeAmin + vtkMath::Dot( pA->Axes[ii], pA->Axes[ii] );
    ...
    if ( eps != 0 )
      { // avoid sqrt call if tolerance check isn't being done
      eps *= sqrt(rangeAmax - rangeAmin);
      }

What confuses me is that the dot product of a vector with itself must always
be a positive number, since such a dot product is equal to x*x + y*y + z*z.
So it looks like there must either be an overflow error or some other 
strange
thing going on.

Adding the fabs() shouldn't hurt anything, but I can't help but wonder
how the negative value occurs.  As a test, do you think you could add

  assert(rangeAmax - rangeAmin >= 0);

to your code and run it through a debugger to see what the values
of rangeAmax and rangeAmin are when it fails?  Just curious.

 - David

Dean Inglis wrote:

>Hi, 
>
>I have submitted the follwoing to the php bug tracker,
>but in the meantime, if someone with extensive knowledge
>of vtkOBBTree could comment on the bug?
>
>I have a fairly complex pipeline that
>cannot readily be encapsulated into a running
>example to demo what I believe is a bug.
>In vtkOBBTree.cxx, at about line 1795, 
>in int vtkOBBTree::LineIntersectsNode()
>the following code can produce a sqrt domain err:
>if ( eps != 0 ) 
>  {
>  eps *= sqrt(rangeAmax - rangeAmin);
>  }
>The errors I encountered are resolved by putting
>the subtraction in a fabs() funtion call within the sqrt.  
>However, I am unsure if this will 
>impact any other pipelines using this class. 
>
>Dean
>
> 
>_______________________________________________
>vtk-developers mailing list
>vtk-developers at vtk.org
>http://www.vtk.org/mailman/listinfo/vtk-developers
>
>  
>