<html>
<body>
<font size=3>Hi Mathieu, Daniel and others,<br><br>
I put together a new SPSA optimizer, based on the three version that we
had. I also added a Test.<br><br>
You may download the zip from:<br>
<a href="http://www.isi.uu.nl/People/Stefan/" eudora="autourl">http://www.isi.uu.nl/People/Stefan/<br><br>
<br>
</a>> - what should be the default values for a, A, C?<br>
> - I used a=A=c=0.1 (Don't trust my
implementation...)<br>
> - Mathieu uses a=01.6, A = 60(!),
C=0.01<br>
> - Stefan uses a=A=c=1.0<br>
> - Are we OK with Alpha = 0.602, Gamma = 0.101?<br><br>
alpha = 0.602 and gamma = 0.101, as J. Spall suggests.<br><br>
c depends on the amount of noise in your problem and the range of your
parameters, so could by anything. So, 1.0 seemed the most appropriate to
me as a default.<br><br>
a depends on your specific problem as well, so is set to 1.0 by
default.<br><br>
A is set to 10.0, which is 10% of the default
MaximumNumberOfIterations.<br><br>
The Scales thing gave me quite a headache but I think I did it correct
now. In the .cxx I tried to explain why I did it like i did...<br><br>
Please let me know what you think about it!<br><br>
Regards,<br>
Stefan.<br><br>
<br>
At 10:04 18/03/05, Mathieu De Craene wrote:<br>
<blockquote type=cite class=cite cite>Le jeudi 17 mars 2005 à 09:10
-0500, Blezek, Daniel J (Research) a<br>
écrit :<br>
> Stefan, Marius, Mathieu,<br>
> <br>
> Excellent. I liked the try / catch blocks, and the
stopping<br>
> criteria in Mathieu's implementation. Essentially, the code is
the<br>
> same across all 3 implementations... As you would expect from
the<br>
> nicely written paper.<br>
> <br>
> - what should be the default values for a, A, C?<br>
> - I used a=A=c=0.1 (Don't trust my
implementation...)<br>
> - Mathieu uses a=01.6, A = 60(!),
C=0.01<br><br>
<br>
In fact, this value should be changed to 0.5 or 1. I used a Set()
method<br>
in the main code to change the default value. For rigid registration
for<br>
instance, I doubt you will measure anything by changing the
translations<br>
coefficients by 0.01 mm ;-)<br><br>
If the A coefficient is big, it means that the optimizer will make
big<br>
steps at the initial iterations.<br>
An idea of the successive trials made by the spsa optimizer in
rigid<br>
registration can be seen on this animation :<br>
<a href="http://www.tele.ucl.ac.be/~decraene/anims/RIGID.mpeg" eudora="autourl">http://www.tele.ucl.ac.be/~decraene/anims/RIGID.mpeg</a><br><br>
I think it is good to keep this parameter big enough if we want to
keep<br>
the robust behavior of this optimizer.<br><br>
> - Stefan uses a=A=c=1.0<br>
> - Are we OK with Alpha = 0.602, Gamma = 0.101?<br>
> <br><br>
I have never experimented other set of values but these work fine
for<br>
me ! <br><br>
> <br>
> -dan<br>
> -----Original
Message-----<br>
> From: Stefan Klein
[<a href="mailto:stefan@isi.uu.nl" eudora="autourl">mailto:stefan@isi.uu.nl</a>]<br>
> Sent: Thursday,
March 17, 2005 8:55 AM<br>
> To: Blezek, Daniel J
(Research); Mathieu De Craene<br>
> Cc:
Insight-users@itk.org; insight-developers@itk.org<br>
> Subject: RE:
[Insight-users] some contributions<br>
> <br>
> <br>
> Eh, but then we'll
have 2 SPSA-optimizers....<br>
> Wouldn't it be the
best to have one that combines the good of<br>
> all 3
versions?<br>
> <br>
> plan:<br>
> - we include the
scales thing and the test and some elements<br>
> from Mathieu's
implementation<br>
> - we'll mail it to
the list.<br>
> - you check it (if
we combined all features correctly) and<br>
> correct it<br>
> - Daniel commits it
to the itk.<br>
> <br>
>
Marius&Stefan.<br>
> <br>
> <br>
> <br>
> At 08:39 17/03/05,
Blezek, Daniel J (Research) wrote:<br>
> > Oddly enough, I
also have an implementation of the SPSA<br>
> >
algorithm! I offered to put Stefan's implementation in,<br>
> > provided he
account for m_Scale and write a test. I'll make<br>
> > the same offer
for you, Mathieu. I've put my code on the<br>
> > itk Wiki @
<a href="http://itk.org/Wiki/SPSAOptimizer" eudora="autourl">http://itk.org/Wiki/SPSAOptimizer</a>
however, I<br>
> > haven't tested my code very well, as it's new.<br>
> > <br>
> > -dan<br>
> > <br>
> > -----Original Message-----<br>
> > From: insight-users-bounces@itk.org<br>
> > [<a href="mailto:insight-users-bounces@itk.org" eudora="autourl">mailto:insight-users-bounces@itk.org</a>]On Behalf Of Mathieu<br>
> > De Craene<br>
> > Sent: Thursday, March 17, 2005 8:20 AM<br>
> > To: Stefan Klein<br>
> > Cc: Insight-users@itk.org; insight-developers@itk.org<br>
> > Subject: Re: [Insight-users] some contributions<br>
> > <br>
> > <br>
> > Le mercredi 16 mars 2005 à 21:34 +0100, Stefan Klein a<br>
> > écrit :<br>
> > > Hi all,<br>
> > > <br>
> > > We have created some classes that are useful for us. We<br>
> > hope also for other <br>
> > > people, so we would like to contribute them. The code can<br>
> > be found for <br>
> > > download on:<br>
> > > <a href="http://www.isi.uu.nl/People/Stefan/" eudora="autourl">http://www.isi.uu.nl/People/Stefan/</a><br>
> > > <br>
> > > - itkEulerTransform.h: This class is a wrap around the<br>
> > Euler2DTransform and <br>
> > > the Euler3DTransform. In this way it is templated over the<br>
> > dimension, like <br>
> > > other Transform-classes.<br>
> > > <br>
> > > - SimultaneousPerturbation.zip: This class implements the<br>
> > so-called <br>
> > > Simultaneous Perturbation method as an ITK optimizer. For<br>
> > more info on this <br>
> > > optimization method, go to:<br>
> > > <a href="http://www.jhuapl.edu/SPSA" eudora="autourl">http://www.jhuapl.edu/SPSA</a><br>
> > > <br>
> > <br>
> > I have also have some code for SPSA optimization. I found it<br>
> > quite<br>
> > useful for rigid registration (it seems to be pretty<br>
> > robust). The source<br>
> > code for a rigid registration application based on ITK is<br>
> > available on <br>
> > <br>
> > <a href="http://euterpe.tele.ucl.ac.be/Waleo2/vesale/" eudora="autourl">http://euterpe.tele.ucl.ac.be/Waleo2/vesale/</a><br>
> > <br>
> > I would like to compare our implementations of this<br>
> > optimizer.<br>
> > <br>
> > Regards,<br>
> > <br>
> > Mathieu.<br>
> > <br>
> > > - FullSearch.zip: This class implements a (semi-)Full<br>
> > Search optimization <br>
> > > routine. This can be useful for research, to scan the<br>
> > optimization surface, <br>
> > > or to evaluate the result of other optimizers. It allows<br>
> > the user to<br>
> > > specify which parameters should be evaluated, in which<br>
> > range.<br>
> > > <br>
> > > Since we have contributed 3 things now, we also have a<br>
> > feature request; as <br>
> > > a reward :-)<br>
> > > <br>
> > > Would it be possible to add a public function to the <br>
> > > MattesMutualInformationImageToImageMetric that forces the<br>
> > metric to select <br>
> > > new spatial samples? This would be useful if you would<br>
> > want new spatial <br>
> > > samples every iteration for example.<br>
> > > <br>
> > > We hope that the contributions are useful. Any comments<br>
> > are welcome!!<br>
> > > <br>
> > > Marius and Stefan.<br>
> > > <br>
> > > <br>
> > > Marius Staring, Stefan Klein<br>
> > > Image Sciences Institute<br>
> > > University Medical Centre Utrecht<br>
> > > Heidelberglaan 100, 3584 CX Utrecht, The Netherlands<br>
> > > phone: +31 (0)30 250 3186, fax: +31 (0)30 251 3399<br>
> > > {marius,stefan}@isi.uu.nl,<br>
> > > <a href="http://www.isi.uu.nl/People/Marius" eudora="autourl">http://www.isi.uu.nl/People/Marius</a><br>
> > > <a href="http://www.isi.uu.nl/People/?stefan" eudora="autourl">http://www.isi.uu.nl/People/?stefan</a><br>
> > > <br>
> ></font></blockquote></body>
<br>
</html>