[vtkusers] Reproducible : vtkBooleanOperationPolyDataFilter for clipping (issue) in VTK 7.1

[Seun Odutola]

Hello everyone,

     I posted few weeks back about a niggling issue I have been facing for a while involving clipping two meshes together using vtkBooleanOperationPolyDataFilter, please see below my previous post. I have been able to write a little application that reproduces this crash easily so that others can review the code and see what needs to be fixed. 
See issue #16954
https://gitlab.kitware.com/vtk/vtk/issues/16954 <https://gitlab.kitware.com/vtk/vtk/issues/16954>


> On Dec 23, 2016, at 12:49 PM, Seun Odutola <seun at rogue-research.com> wrote:
> 
> Hello everyone,
> 
>     I have a reproducible case where I am trying to clip two meshes, when the function vtkBooleanOperationPolyDataFilter::RequestData is executed its call to SortPolyData fails triggering a crash trying to access the first index of polydata’s cell (see vtkBooleanOperationPolyDataFilter::SortPolyData line 62). On tracing the cause of the crash, I noticed that the polydata’s cell data array is null and there seems to be no guard against trying to probe its value, secondly prior to calling SortPolyData, two vtkPolyData’s pd0 & pd1 of which the latter possess no poly and both have their cells and links generated but the latter still lacks poly. I decided to introduce a condition just before building the cells and links for both pd0 and pd1 to check if either have no poly and if so exit the function (return 0), doing so completely prevents my program from crashing. I am not sure if this is the right approach nor if adding the check is necessary.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtkusers/attachments/20170113/1d699a4f/attachment.html>