[Insight-users] Re: [Insight-developers] a small bug initkConjugateGradientOptimizer

Wed Jun 22 17:33:38 EDT 2005

Thanks to Jordan, Einstein, Karthik and Zachary,
 For the time being I am trying one plus one evolutionary optimizer and 
trying to read a bit on it too.
 I really appreciate the help !
with regards,
Ashish.

 On 6/22/05, Zachary Pincus <zpincus at stanford.edu> wrote: 
> 
> Also, the OnePlusOneEvolutionaryOptimizer (
> http://itk.org/Insight/Doxygen/html/
> classitk_1_1OnePlusOneEvolutionaryOptimizer.html ) is an optimizer with
> a simulated annealing flavor. It takes random steps within a certain
> radius in parameter space, and adjusts that radius (anisotropically) to
> selectively favor directions which have produced good results in the
> past. With each "good" step, the radius grows, and with each "bad" step
> it shrinks (akin to the "cooling" that you do toward the end of a
> simulated annealing run).
> 
> Zach
> 
> On Jun 22, 2005, at 2:08 PM, Karthik Krishnan wrote:
> 
> >
> >
> > Einstein, Daniel R wrote:
> >
> >> Anish,
> >> As far as I can tell, all of the optimization algorithms from Netlib
> >> are local. Global optimization is considerably harder, and requires
> >> much more crunching. Examples are simulated annealing, multi-starts,
> >> i.e. stochastically sampling the solutions space, particle swarm
> >> methods, and sequential response surfaces. I am new enough to ITK
> >> that I cannot say which if any of these might usefully be implemented
> >> in ITK.
> >
> > http://itk.org/Insight/Doxygen/html/classitk_1_1SPSAOptimizer.html was
> > recently added
> >
> >> Particle swarm methods are an interesting option because they are so
> >> easy to program. Be advised, however, there are no global methods
> >> that guarantee conversion.
> >> Dan
> >>
> >> Daniel R Einstein, PhD
> >> Biological Monitoring and Modeling
> >> Pacific Northwest National Laboratory
> >> P.O. Box 999; MSIN P7-59
> >> Richland, WA 99352
> >> Tel: 509/ 376-2924
> >> Fax: 509/376-9064
> >> _daniel.einstein at pnl.gov_ <mailto:daniel.einstein at pnl.gov>
> >>
> >>
> >> ----------------------------------------------------------------------
> >> --
> >> *From:* insight-users-bounces+daniel.einstein=pnl.gov at itk.org
> >> [mailto:insight-users-bounces+daniel.einstein=pnl.gov at itk.org] *On
> >> Behalf Of *Ashish Poddar
> >> *Sent:* Wednesday, June 22, 2005 10:58 AM
> >> *To:* Luis Ibanez
> >> *Cc:* insight-users @ itk. org
> >> *Subject:* [Insight-users] Re: [Insight-developers] a small bug
> >> initkConjugateGradientOptimizer
> >>
> >> Hi,
> >> Most of the optimizers which I have came across help in finding the
> >> local minima around the given initial approximation. But in that case
> >> I mostly end up in a wrong place. Is there any algorithm which helps
> >> to scan the global space somehow and help in determining the global
> >> minima.
> >> Other problem that I am facing with the conjugate gradient method is
> >> that the scales do not work for conjugate gradient optimizer. The
> >> scales are only taken into account in the very first initialization
> >> step and are never considered again in any of the latter iterations.
> >> I want to fix some of the parameters by setting the scale to some
> >> extreme value (I used to set 100 or something for regular step
> >> gradient descent optimizer and it used to serve the purpose very
> >> conveniently).
> >> any help will be highly appreciated,
> >> with regards,
> >> Ashish.
> >>
> >> On 5/18/05, *Luis Ibanez* <luis.ibanez at kitware.com
> >> <mailto:luis.ibanez at kitware.com>> wrote:
> >>
> >>
> >> Hi Ashish,
> >>
> >> The Conjugate Gradient method is only convenient when the cost
> >> function has smooth second derivatives. If your cost function
> >> is noisy, is is unlikely that this optimizer will behave nicely.
> >>
> >> Note that is is common to find that Image Metrics are rather
> >> noisy functions.
> >>
> >>
> >>
> >> Regards,
> >>
> >>
> >> Luis
> >>
> >>
> >>
> >> --------------------
> >>
> >> Ashish Poddar wrote:
> >>
> >> > Hi,
> >> >
> >> > I also am right now struggling with the initialization options
> >> for the
> >> > Conjugate Gradient for which I could not find any examples.
> >> While
> >> > searching I came across an example for Levenberg Marquardt
> >> Optimizer
> >> > which seems to be having similar interface as that of conjugate
> >> > gradient optimizer. However the similar initialization did not
> >> worked
> >> > for Conjugate gradient. If someone can point out any reference
> >> for
> >> > Conjugate Gradient, it would be great.
> >> >
> >> > Earlier I was using regular step gradient descent optimizer with
> >> these
> >> > parameters:
> >> > Transform - Centered Affine
> >> > Scale for first 9 parameters - 1.0
> >> > Scale for next 6 parameters - 0.0001
> >> > Number of Iterations - 400
> >> > Minimum Step Length - 0.0001
> >> > Maximum Step Length - 0.005
> >> >
> >> > Any help will be highly appreciated,
> >> > with regards,
> >> > Ashish.
> >> >
> >> >
> >> >
> >> > On 5/17/05, Ashish Poddar <ahpoddar at gmail.com
> >> <mailto:ahpoddar at gmail.com>> wrote:
> >> >
> >> >>Hi Luis,
> >> >>
> >> >>Thank you for the quick action. probably similar change is
> >> required
> >> >>for Levenberg Marquardt Optimizer too.
> >> >>
> >> >>with regards,
> >> >>Ashish.
> >> >>
> >> >>On 5/16/05, Luis Ibanez <luis.ibanez at kitware.com
> >> <mailto:luis.ibanez at kitware.com>> wrote:
> >> >>
> >> >>>Hi Ashish,
> >> >>>
> >> >>>Thanks for pointing this out.
> >> >>>
> >> >>>You are right, the GetValue() method should be const.
> >> >>>
> >> >>>A fix has now been committed to the CVS repository.
> >> >>>
> >> >>>Please let us know if you encounter any other problem.
> >> >>>
> >> >>> Thanks
> >> >>>
> >> >>> Luis
> >> >>>
> >> >>>----------------------
> >> >>>Ashish Poddar wrote:
> >> >>>
> >> >>>>hi,
> >> >>>>
> >> >>>>I am not sure whether it qualifies as a bug or not, but surely
> >> affects
> >> >>>>the re-usability and pluggability model of ITK Library.
> >> >>>>
> >> >>>>the GetValue() function in ConjugateGradientOptimizer class
> >> currently is
> >> >>>>
> >> >>>>MeasureType GetValue();
> >> >>>>
> >> >>>>but in case of RegularStepGradientDescentOptimizer class its
> >> defined by macro as
> >> >>>>
> >> >>>>MeasureType GetValue() const;
> >> >>>>
> >> >>>>which is an interface mis-match... This I encountered when i
> >> replaced
> >> >>>>regular step gradient descent optimizer by conjugate gradient
> >> >>>>optimizer. In the observer I was using a const reference of
> >> the
> >> >>>>optimizer and displaying the value (just the example which is
> >> >>>>available for the same nothing new =D)...
> >> >>>>
> >> >>>>with regards,
> >> >>>>Ashish.
> >> >>>>
> >> >>>
> >> >>>
> >> >>--
> >> >>Ashish Poddar
> >> >>Have an acceptable reason for accepting anything.
> >> >>Y:ashish_poddar | MSN:ashish_poddar at yahoo.com
> >> <mailto:MSN:ashish_poddar at yahoo.com>
> >> >>
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Ashish Poddar
> >> Have an acceptable reason for accepting anything.
> >> Y:ashish_poddar | MSN:ashish_poddar at yahoo.com
> >> <mailto:MSN:ashish_poddar at yahoo.com>
> >>
> >> ----------------------------------------------------------------------
> >> --
> >>
> >> _______________________________________________
> >> Insight-users mailing list
> >> Insight-users at itk.org
> >> http://www.itk.org/mailman/listinfo/insight-users
> >>
> > _______________________________________________
> > Insight-users mailing list
> > Insight-users at itk.org
> > http://www.itk.org/mailman/listinfo/insight-users
> >
> 
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org
> http://www.itk.org/mailman/listinfo/insight-users
> 

-- 
Ashish Poddar
Have an acceptable reason for accepting anything.
Y:ashish_poddar | MSN:ashish_poddar at yahoo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://public.kitware.com/pipermail/insight-users/attachments/20050622/3ae409b1/attachment.html