[vtk-developers] Checking SMP backend at runtime/compiletime?

Tue Feb 9 14:55:37 EST 2016

A few comments:

* I support the path David wants to take. We flushed this out over several
months in collaboration with a Google Summer of Code student. It is the
best transition strategy given that we have several moving components (see
below).

* The Simple backend of vtkSMPTools is only there for debugging. Helgrind
produces lots of false positives when using TBB so I developed the Simple
backend for use with Helgrind. It is not a production backend and pretty
much sucks. Now that we haven an OpenMP backend that can be used with
Helgrind, Simple must die. I don't see a reason to deprecate it first since
it is there only for debugging. This is clearly documented in the PDF will
pointed to.

* For compilers that do not support OpenMP 3.1, one can (and should) use
TBB. TBB is the better backend anyway so I recommend it over OpenMP.

* We will not include TBB in VTK. It is an external dependency similarly to
OpenGL & MPI. In the future, folks will have to get it or have OpenMP if
they want any thread-level parallelism out of VTK. We need to discuss what
"in the future" means.

* Posix threads, C++11 threads etc. are not the way to go. They are way too
low level and require management of thread pools and such to get good
scalability. Things that OpenMP and TBB already to well. In general, for
the kind of parallel computing we want in VTK, the best tools are high
level ones such as parallel for loops etc. Furthermore, OpenMP will be
important where we want to get SIMD (vector) parallelism.
Auto-vectorization is very imperfect. And there are no C++ primitives that
help with SIMD in C++11.

* At one point, we will have to get rid of vtkMultiThreader (at least of
its use in algorithms, it may still be useful for GUI threads and whatnot).
Hopefully, by then OpenMP 3.1 or above will be universally supported so we
can make it the default backend. If not though, we'll have to require that
people use TBB.

Best,
-berk

On Tue, Feb 9, 2016 at 11:33 AM, Ken Martin <ken.martin at kitware.com> wrote:

> Echoing David's earlier comments it would seem like we would want a nice
> path to convert existing multithreaded algorithms to use vtkSMPTools
> knowing that vtkSMPTools would not slow down the existing algorithm. Doing
>
> #if VTK_SMP_BACKEND == SLOW
>  use vtkMultithreader
> #else
>   use vtkSMPTools
> #endif
>
> sounds odd. I did not read the pdf so if that is covered in there
> apologies.
>
>
>
>
> On Tue, Feb 9, 2016 at 10:56 AM, David Gobbi <david.gobbi at gmail.com>
> wrote:
>
>> Hi Sujin,
>>
>> That sounds good.  Even if the choice of backend is transparent as far as
>> using vtkSMPTools is concerned, it's very nice to be able to report which
>> backend was configured.
>>
>>  - David
>>
>> On Tue, Feb 9, 2016 at 8:46 AM, Sujin Philip <sujin.philip at kitware.com>
>> wrote:
>>
>>> Hi Sean,
>>>
>>> vtkSMPTools is a framework for implementing multi-threaded algorithms in
>>> VTK. It support several backends. The main ones are TBB and OpenMP. There
>>> are Kaapi and Simple backends which are no longer supported and will be
>>> removed soon. Finally, the default backend is Sequential which is just a
>>> single threaded implementation of the framework. After removal of the Kaapi
>>> and Simple backend, if you need multithreading support on Clang you would
>>> have to use TBB. The Sequential backend will be supported on all platforms.
>>>
>>> David,
>>>
>>> I have talked with Berk about this and I will soon make a change to have
>>> a compile time macro to check for SMP backend type. I will also finally
>>> remove Kaapi and Simple backend as part of this change.
>>>
>>> Thanks
>>> Sujin
>>>
>>>
>>> On Tue, Feb 9, 2016 at 10:28 AM, Sean McBride <sean at rogue-research.com>
>>> wrote:
>>>
>>>> On Tue, 9 Feb 2016 09:13:48 -0500, Sujin Philip said:
>>>>
>>>> >Why would you want to continue using vtkMultiThreader when Sequential
>>>> or
>>>> >Simple is used? In fact, now that there is an openmp backend, we
>>>> should be
>>>> >removing simple. It was only there to ease debugging since tbb had very
>>>> >complex back-traces. Openmp back-traces are much more readable. Do you
>>>> want
>>>> >the algorithm to be multithreaded even when Sequential is used?
>>>>
>>>> I don't know the APIs you're discussing, so this comment is coming
>>>> mostly from ignorance, but: are you talking about requiring OpenMP to build
>>>> VTK?  Clang has only very recently added OpenMP support, and IIRC it's not
>>>> complete.  Also, last I checked, Apple's fork of clang doesn't support it
>>>> at all.
>>>>
>>>> Cheers,
>>>>
>>>> --
>>>> ____________________________________________________________
>>>> Sean McBride, B. Eng                 sean at rogue-research.com
>>>> Rogue Research                        www.rogue-research.com
>>>> Mac Software Developer              Montréal, Québec, Canada
>>>>
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>>
>
>
> --
> Ken Martin PhD
> Chairman & CFO
> Kitware Inc.
> 28 Corporate Drive
> Clifton Park NY 12065
> 518 371 3971
>
> This communication, including all attachments, contains confidential and
> legally privileged information, and it is intended only for the use of the
> addressee.  Access to this email by anyone else is unauthorized. If you are
> not the intended recipient, any disclosure, copying, distribution or any
> action taken in reliance on it is prohibited and may be unlawful. If you
> received this communication in error please notify us immediately and
> destroy the original message.  Thank you.
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/vtk-developers/attachments/20160209/72c14860/attachment.html>