[vtk-developers] Integer overflow when clipping large volumes

[Hahn, Steven E.]

I encountered a segmentation fault when attempting to clip a stack of tiffs (vtkImageData) in ParaView and, using the undefined behavior sanitizer, tracked it down to integer overflow in vtkTableBasedClipDataSet. Changing several variables from int to vtkIdType seems to correct the problem.

I opened a merge request with these changes and a python file to reproduce the issue. The primary contributor to vtkTableBasedClipDataSet.cxx isn’t on the GitLab site, so I’m not sure who to tag. Is someone else interested in reviewing it?

https://gitlab.kitware.com/vtk/vtk/merge_requests/4620

Steven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://public.kitware.com/pipermail/vtk-developers/attachments/20180824/0d9649d1/attachment.html>