[Insight-users] Itk 4 - Multithreading over multiple nodes of linux high performance computer

[Johnson, Hans J]

Bilal,

Unfortunately the answer is that you can not take advantage of multiple nodes or OpenMP.  The reality is that in many cases ITK does not efficiently take advantage of the parallelism that is available in modern multi-core computers.  My experience is that it often scales well to 4 processors, but  rarely takes full advantage of the resources for 16 processors.  This is an ongoing area of improvement.

Of particular interest is the antsRegistration example. I have been working on improving the performance of this tool in these patch sets http://review.source.kitware.com/#/q/status:open+project:ITK+branch:master+topic:RemoveFalseSharing,n,z

In some cases I have been able to improve the perforce by 70x by identifying the ways to better utilize the existing threads.  It is my belief that you would benefit more from helping improve the utilization of the existing threads than to try to add more threads across different platforms.

Regards,
Hans

From: Bilal Tahir <bilalahmedtahir at googlemail.com<mailto:bilalahmedtahir at googlemail.com>>
Date: Thursday, October 3, 2013 8:50 AM
To: "insight-users at itk.org<mailto:insight-users at itk.org>" <insight-users at itk.org<mailto:insight-users at itk.org>>
Subject: [Insight-users] Itk 4 - Multithreading over multiple nodes of linux high performance computer


Dear ITK community,

This is my first email on this mail base so please bear with me.

My query is regarding multithreading over multiple cores. In our high performance Linux Sun Grid Engine, we have 16 CPUs per node. Currently I use image registration software based on ITK 4 (e.g. ants Registration,  Elastix)  and set the environment variable ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS and openMP options in my bash scripts to 16.

However, some of our Linux administrators suggested to run parallel across multiple nodes to increase the total number of CPUs. Is this possible with ITK 4 and does it require something like the MPI framework? Without some form of message passing I would imagine it will be difficult to envisage taking advantage of multiple nodes from a single program. I heard there were some discussions about MPI implementations of ITK 4 but nothing has materialised yet.

Could anyone shed any light as to whether it is feasible to run ITK 4 over several nodes and how to go about doing this?

Best wishes

Bilal


________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged.  If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited.  Please reply to the sender that you have received the message in error, then delete it.  Thank you.
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/pipermail/insight-users/attachments/20131003/0bdf0efa/attachment.htm>