[Insight-developers] suggestion: use environment variable to limit number of parallel threads

[Torsten Rohlfing]

Hi Bill --

I usually use command line arguments in my own tools as well, but I'd 
propose that this is something you'd like to control even when you 
didn't write the tool you're using, especially in cases where you may 
not have the source code. In the latter case, there is currently no way 
of telling an ITK-based program to play nicely with shared resources.

At the very least, using command line argument seems like unnecessary 
replication of code.

Can you think of another way of achieving the effect I am suggesting, 
one that doesn't use environment variables?

I guess one could run such tools in a virtual machine that pretends to 
have fewer CPUs than the physical one, but that seems like a bit of 
overkill.

Cheers!

TR

Bill Lorensen wrote:
> Torsten,
>  
> At GE (when I was there), we passed the number of threads as a command 
> line argument. We had the same issues that you have. In general, I 
> don't think we use environment variables in ITK although I can't 
> recall the justification.
>  
> Bill
>
>  
> On 9/19/07, *Torsten Rohlfing* <torsten at synapse.sri.com 
> <mailto:torsten at synapse.sri.com>> wrote:
>
>     Hi:
>
>     I have a suggestion for ITK's multi-threaded processing framework. I
>     think it would be useful to be able to set the maximum number of
>     parallel threads using an environment variable, for example using
>
>     export ITK_MAXIMUM_NUMBER_OF_THREADS=4
>
>     (other shells and Windows accordingly).
>
>     This would solve a problem I am having when running jobs with
>     ITK-based
>     applications in batch mode on my cluster. It is not always
>     beneficial to
>     run on all available CPUs, because other steps in a processing
>     pipeline
>     may not be parallel, and it may therefore be desirable to run
>     multiple
>     copies of the same pipeline on different data instead.
>
>     By implementing this through an environment variable, we'd have this
>     functionality in every application that uses ITK, without for example
>     having to put a separate command line switch into each and every
>     one of
>     them.
>
>     Also, the solution would be backward compatible, because if the chosen
>     variable is not set, ITK can simply behave as it does now and allocate
>     one thread per CPU.
>
>     Best,
>     Torsten
>
>     --
>     Torsten Rohlfing, PhD          SRI International, Neuroscience Program
>     Research Scientist             333 Ravenswood Ave, Menlo Park, CA
>     94025
>     Phone: ++1 (650) 859-3379      Fax: ++1 (650) 859-2743
>        torsten at synapse.sri.com
>     <mailto:torsten at synapse.sri.com>        http://www.stanford.edu/~rohlfing/
>     <http://www.stanford.edu/%7Erohlfing/>
>
>         "Though this be madness, yet there is a method in't"
>
>
>     _______________________________________________
>     Insight-developers mailing list
>     Insight-developers at itk.org <mailto:Insight-developers at itk.org>
>     http://www.itk.org/mailman/listinfo/insight-developers
>     <http://www.itk.org/mailman/listinfo/insight-developers>
>
>
>

-- 
Torsten Rohlfing, PhD          SRI International, Neuroscience Program
 Research Scientist             333 Ravenswood Ave, Menlo Park, CA 94025
  Phone: ++1 (650) 859-3379      Fax: ++1 (650) 859-2743
   torsten at synapse.sri.com        http://www.stanford.edu/~rohlfing/

     "Though this be madness, yet there is a method in't"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: torsten.vcf
Type: text/x-vcard
Size: 366 bytes
Desc: not available
Url : http://www.itk.org/mailman/private/insight-developers/attachments/20070919/4297ffd0/torsten.vcf