[Insight-users] how to temparaly disable multithreading or limit the # of threads

[pinpress]

Hi Luis,

Here are the list of answers to the list of questions:

a). Both fixed and moving images were 256-by-256-by-124, MR images;
b). # of samples in the metric was 2% of the total # of voxels (1.6253e+005)
c). # of bins: 256 (as images were 8-bit gray scale);
d). I am not sure -- the program was based on Registration example
DeformableRegistration14.cxx, which uses Mattes MI, and
itkBSplineDeformableTransform. But I guess it did use optimized method, as
at one time I had some error message:

File:
/PATH/apps/itk/include/InsightToolkit/Review/itkOptMattesMutualInformationImageToImageMetric.txx
Line: 1034
Description: itk::ERROR:
MattesMutualInformationImageToImageMetric(0x796150): Joint PDF summed to
zero

By the way, the above error was caused by the fact that I used Analyze image
format with the fixed image (on Linux platform). If I changed the image
format to binary VTK, the program worked without the above error. What
really perplexed me, though, was that on my Windows computer, it was the
other way around: If I used VTK in the fixed image, it reports similar error
saying that too many samples out side the mapped region. But if I used
Analyze data format, then the program works fine. 

Note that both Analyze or binary VTK format were for exactly the same image,
and they both used "row major" when writing the image data (as opposed to
"column major" as in Matlab, from which Analyze and VTK images were
written). If really there was something wrong with writing the data to
files, then both Linux and Windows should report the same error, right? But
apparently it was not the case, which caused me to think whether Linux and
Windows follow the same image format, e.g., both use "row major"??

e). Yes, I did in fact set the SetUseExplicitPDFDerivatives to "1".
f). Yes,  I did in fact set SetUseCachingOfBSplineWeights to "1" as well.

And finally, I had 8G RAM on both Linux and Winodws, both machines are
64-bit (Ubuntu and Vista). 

Now it is clear that I had overlooked the setting in (e) above: if I turn it
off, things are much improved. 

Here are some new updates (timing only reflect the registration, stripping
all other parts):

# of threads          time
1                         58''
2                         33''
8                         17'' (using all CPUs will not kill my computer)

Very nice result. In addition, much less memory was needed (<4%).

One last question: what multithreading scheme does ITK use? It does not seem
to be based on OpenMP, as I tried to set the "OMP_NUM_THREADS" environment
variable earlier and it did not work.

Thanks very much!

Pinpress

P.S. Also thanks for your youtube link: it was really hilarious!



Luis Ibanez wrote:
> 
> Hi Pinpress.
> 
> Yeap, you have encountered the classical trade-off between memory
> and speed.
> 
> When using multiple threads, there is a need for replication of
> data. Although the ration shouldn't be quite as linear as you
> seem to be observing.
> 
> Could you please provide specific numbers ?
> 
> In particular:
> 
> a) Image size in pixels ?
> b) Number of samples in the Metric ?
> c) Number of bins in the Metric ?
> d) Are you using the Optimized Registratin methods in the Code/Review
>     directory ?
> e) Do you have the flag for explicit PDF derivatives ON ?
> f) Do you have the flag for caching terms of the jacobian ON ?
> 
> How much RAM do you actually have in this machine.
> 
> Did it ever started swapping memory to disk ?
> 
> 
> The correctness of your conclusions about scalability is subject
> to the verification that the capabilities of your machine match
> the memory requirements of the problem.
> 
> In other words, if the settings of the numbers above is such that
> you end up requiring too much memory from your computer, then the
> poor performance of the algorithm at high number of threads may
> not be attributed directly to the scalability of the algorithm,
> but to the actual limitations of the hardware.
> 
> Please let us know about the specific numbers above, so that
> we can arrive to conclusions based on facts.
> 
> 
>     Regards,
> 
> 
>        Luis
> 
> 
> 
> -----
> 
> 
> PS. For the humorous note, your statement of
> 
>           "I did not go beyond 4 threads,
>            as it may cause my computer to die"
> 
>       reminds me of Demetri Martin's joke about batteries:
> 
>         "Batteries are the most dramatic objects...
>          other things stop working or the break
>          But batteries..
>          THEY DIE".
> 
>       http://www.youtube.com/watch?v=XiFrfeJ8dKM
> 
-- 
View this message in context: http://www.nabble.com/how-to-temparaly-disable-multithreading-or-limit-the---of-threads-tp20548844p20561092.html
Sent from the ITK - Users mailing list archive at Nabble.com.