[Insight-users] Itk 4 - Multithreading over multiple nodes of linux high performance computer

[Matt McCormick]

PS Another good link on contributing via reviews:

  http://insightsoftwareconsortium.github.io/ITKBarCamp-doc/CommunitySoftwareProcess/PerformAGerritReview/index.html

On Thu, Oct 3, 2013 at 4:07 PM, Matt McCormick
<matt.mccormick at kitware.com> wrote:
> Hi Bilal,
>
> We have started to merge these patches, and we will continue to merge
> them in the coming weeks.  Instruction for downloading the Git version
> can be found here:
>
>   http://www.itk.org/Wiki/ITK/Git/Download
>
> To get involved in the patch review and development, it would be good
> to join the code review hangouts announced on the ITK Bar Camp G+
> Community:
>
>  https://plus.google.com/communities/111375098792764998322/stream/a1cf9df1-06f5-4abd-9142-bd8149f6b191
>
> Thanks,
> Matt
>
> On Thu, Oct 3, 2013 at 2:21 PM, Bilal Tahir
> <bilalahmedtahir at googlemail.com> wrote:
>> Dear Hans,
>>
>> Many thanks for your reply. I would like to try your patches for
>> antsRegistration to see if it speeds up my pipelines using the SyN
>> algorithm. I am new to the process of applying patches so any guidance on
>> what steps are required would be greatly appreciated.
>>
>> Best wishes
>>
>> Bilal
>>
>> On 3 Oct 2013 14:58, "Johnson, Hans J" <hans-johnson at uiowa.edu> wrote:
>>>
>>> Bilal,
>>>
>>> Unfortunately the answer is that you can not take advantage of multiple
>>> nodes or OpenMP.  The reality is that in many cases ITK does not efficiently
>>> take advantage of the parallelism that is available in modern multi-core
>>> computers.  My experience is that it often scales well to 4 processors, but
>>> rarely takes full advantage of the resources for 16 processors.  This is an
>>> ongoing area of improvement.
>>>
>>> Of particular interest is the antsRegistration example. I have been
>>> working on improving the performance of this tool in these patch sets
>>> http://review.source.kitware.com/#/q/status:open+project:ITK+branch:master+topic:RemoveFalseSharing,n,z
>>>
>>> In some cases I have been able to improve the perforce by 70x by
>>> identifying the ways to better utilize the existing threads.  It is my
>>> belief that you would benefit more from helping improve the utilization of
>>> the existing threads than to try to add more threads across different
>>> platforms.
>>>
>>> Regards,
>>> Hans
>>>
>>> From: Bilal Tahir <bilalahmedtahir at googlemail.com>
>>> Date: Thursday, October 3, 2013 8:50 AM
>>> To: "insight-users at itk.org" <insight-users at itk.org>
>>> Subject: [Insight-users] Itk 4 - Multithreading over multiple nodes of
>>> linux high performance computer
>>>
>>> Dear ITK community,
>>>
>>> This is my first email on this mail base so please bear with me.
>>>
>>> My query is regarding multithreading over multiple cores. In our high
>>> performance Linux Sun Grid Engine, we have 16 CPUs per node. Currently I use
>>> image registration software based on ITK 4 (e.g. ants Registration,
>>> Elastix)  and set the environment variable
>>> ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS and openMP options in my bash scripts
>>> to 16.
>>>
>>> However, some of our Linux administrators suggested to run parallel across
>>> multiple nodes to increase the total number of CPUs. Is this possible with
>>> ITK 4 and does it require something like the MPI framework? Without some
>>> form of message passing I would imagine it will be difficult to envisage
>>> taking advantage of multiple nodes from a single program. I heard there were
>>> some discussions about MPI implementations of ITK 4 but nothing has
>>> materialised yet.
>>>
>>> Could anyone shed any light as to whether it is feasible to run ITK 4 over
>>> several nodes and how to go about doing this?
>>>
>>> Best wishes
>>>
>>> Bilal
>>>
>>>
>>>
>>> ________________________________
>>> Notice: This UI Health Care e-mail (including attachments) is covered by
>>> the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is
>>> confidential and may be legally privileged.  If you are not the intended
>>> recipient, you are hereby notified that any retention, dissemination,
>>> distribution, or copying of this communication is strictly prohibited.
>>> Please reply to the sender that you have received the message in error, then
>>> delete it.  Thank you.
>>> ________________________________
>>
>>
>> _____________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://www.kitware.com/products/protraining.php
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-users
>>