[vtk-developers] Undefined Behavior in vtkPointLocator.h

[Mmanu Chaturvedi]

Hi!

On building VTK with UBSan, and running the following test:

`ctest -VV -R vtkInteractionWidgetsCxx-TestPlaneWidget`

UBSan produces runtime errors due to the casting done in lines 282 --- 284,
similar to this:

runtime error: value -1.81708e+299 is outside the range of representable
values of type 'long long'

I believe one way to solve the problem would be to do something like the
following:
https://gitlab.kitware.com/m-chaturvedi/vtk/commit/4df07e6d3b9bc9250bab74287fb2df3fe1d2098d

But this increases the test's time significantly.

I was wondering if someone could propose a proper fix.

vtkTypeId --
https://gitlab.kitware.com/m-chaturvedi/vtk/blob/master/Common/Core/vtkType.h#L328

Thanks,
Mmanu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://vtk.org/pipermail/vtk-developers/attachments/20180110/3ca1eece/attachment.html>