[Insight-users] Re: [Insight-developers] a small bug initkConjugateGradientOptimizer

[Einstein, Daniel R]

Anish,
 
As far as I can tell, all of the optimization algorithms from Netlib are
local. Global optimization is considerably harder, and requires much
more crunching. Examples are simulated annealing, multi-starts, i.e.
stochastically sampling the solutions space, particle swarm methods, and
sequential response surfaces. I am new enough to ITK that I cannot say
which if any of these might usefully be implemented in ITK. Particle
swarm methods are an interesting option because they are so easy to
program. Be advised, however, there are no global methods that guarantee
conversion.
 
Dan
 

Daniel R Einstein, PhD
Biological Monitoring and Modeling
Pacific Northwest National Laboratory
P.O. Box 999; MSIN P7-59
Richland, WA 99352
Tel: 509/ 376-2924
Fax: 509/376-9064
daniel.einstein at pnl.gov <mailto:daniel.einstein at pnl.gov>  

 

________________________________

From: insight-users-bounces+daniel.einstein=pnl.gov at itk.org
[mailto:insight-users-bounces+daniel.einstein=pnl.gov at itk.org] On Behalf
Of Ashish Poddar
Sent: Wednesday, June 22, 2005 10:58 AM
To: Luis Ibanez
Cc: insight-users @ itk. org
Subject: [Insight-users] Re: [Insight-developers] a small bug
initkConjugateGradientOptimizer


Hi,
 
Most of the optimizers which I have came across help in finding the
local minima around the given initial approximation. But in that case I
mostly end up in a wrong place. Is there any algorithm which helps to
scan the global space somehow and help in determining the global minima.

 
Other problem that I am facing with the conjugate gradient method is
that the scales do not work for conjugate gradient optimizer. The scales
are only taken into account in the very first initialization step and
are never considered again in any of the latter iterations. I want to
fix some of the parameters by setting the scale to some extreme value (I
used to set 100 or something for regular step gradient descent optimizer
and it used to serve the purpose very conveniently). 
 
any help will be highly appreciated,
with regards,
Ashish.

 
On 5/18/05, Luis Ibanez <luis.ibanez at kitware.com> wrote: 


	Hi Ashish,
	
	The Conjugate Gradient method is only convenient when the cost
	function has smooth second derivatives.  If your cost function 
	is noisy, is is unlikely that this optimizer will behave nicely.
	
	Note that is is common to find that Image Metrics are rather
	noisy functions.
	
	
	
	   Regards,
	
	
	      Luis
	
	
	
	--------------------
	
	Ashish Poddar wrote:
	
	> Hi,
	>
	> I also am right now struggling with the initialization options
for the
	> Conjugate Gradient for which I could not find any examples.
While 
	> searching I came across an example for Levenberg Marquardt
Optimizer
	> which seems to be having similar interface as that of
conjugate
	> gradient optimizer. However the similar initialization did not
worked 
	> for Conjugate gradient. If someone can point out any reference
for
	> Conjugate Gradient, it would be great.
	>
	> Earlier I was using regular step gradient descent optimizer
with these
	> parameters: 
	> Transform - Centered Affine
	> Scale for first 9 parameters - 1.0
	> Scale for next 6 parameters - 0.0001
	> Number of Iterations - 400
	> Minimum Step Length - 0.0001
	> Maximum Step Length - 0.005
	>
	> Any help will be highly appreciated,
	> with regards,
	> Ashish.
	>
	>
	>
	> On 5/17/05, Ashish Poddar <ahpoddar at gmail.com> wrote: 
	>
	>>Hi Luis,
	>>
	>>Thank you for the quick action. probably similar change is
required
	>>for Levenberg Marquardt Optimizer too.
	>>
	>>with regards,
	>>Ashish. 
	>>
	>>On 5/16/05, Luis Ibanez <luis.ibanez at kitware.com> wrote:
	>>
	>>>Hi Ashish,
	>>>
	>>>Thanks for pointing this out. 
	>>>
	>>>You are right, the GetValue() method should be const.
	>>>
	>>>A fix has now been committed to the CVS repository.
	>>>
	>>>Please let us know if you encounter any other problem. 
	>>>
	>>>    Thanks
	>>>
	>>>       Luis
	>>>
	>>>----------------------
	>>>Ashish Poddar wrote:
	>>>
	>>>>hi,
	>>>> 
	>>>>I am not sure whether it qualifies as a bug or not, but
surely affects
	>>>>the re-usability and pluggability model of ITK Library.
	>>>>
	>>>>the GetValue() function in ConjugateGradientOptimizer class
currently is 
	>>>>
	>>>>MeasureType GetValue();
	>>>>
	>>>>but in case of RegularStepGradientDescentOptimizer class its
defined by macro as
	>>>>
	>>>>MeasureType GetValue() const; 
	>>>>
	>>>>which is an interface mis-match... This I encountered when i
replaced
	>>>>regular step gradient descent optimizer by conjugate
gradient
	>>>>optimizer. In the observer I was using a const reference of
the 
	>>>>optimizer and displaying the value (just the example which
is
	>>>>available for the same nothing new =D)...
	>>>>
	>>>>with regards,
	>>>>Ashish. 
	>>>>
	>>>
	>>>
	>>--
	>>Ashish Poddar
	>>Have an acceptable reason for accepting anything.
	>>Y:ashish_poddar | MSN:ashish_poddar at yahoo.com
	>>
	>
	>
	>
	
	
	
	




-- 
Ashish Poddar
Have an acceptable reason for accepting anything.
Y:ashish_poddar | MSN:ashish_poddar at yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/insight-users/attachments/20050622/17f71711/attachment-0001.html