<html>
<body>
<font size=3>Hi,<br><br>
<br>
<blockquote type=cite class=cite cite>If we refer to the classical
gradient descent optimizer,<br>
<a href="http://www.itk.org/cgi-bin/viewcvs.cgi/Code/Numerics/itkGradientDescentOptimizer.cxx?root=Insight&view=markup" eudora="autourl">http://www.itk.org/cgi-bin/viewcvs.cgi/Code/Numerics/itkGradientDescentOptimizer.cxx?root=Insight&view=markup</a><br>
it seems there is to phases. In the first one, the gradient is
computed<br>
without taking into account the scales<br>
<x-tab> </x-tab>m_CostFunction->GetValueAndDerivative(
<br>
<x-tab> </x-tab>this->GetCurrentPosition(),
m_Value, m_Gradient );<br>
In the second phase, the scales are applied to the gradient before<br>
updating the parameters<br>
<x-tab> </x-tab>transformedGradient[j]
= m_Gradient[j] / scales[j];<br>
</font></blockquote><br>
Yep, that was also my first thought. In my first implementation I did it
like this.<br><br>
<blockquote type=cite class=cite cite><font size=3>In your
implementation, you take into account the scales in the first<br>
phase (gradient computation) when applying the perturbation <br>
<x-tab> </x-tab>m_Delta[j]
/= sqrt(scales[j]);<br>
</font></blockquote><br>
This is what Daniel did in his implementation. And I think it is correct.
Take for example the rigid registration problem. Increasing the rotation
with an angle of 0.1 rad would have a big influence on the similarity
measure. However, increasing the x-translation with 0.1 mm would only
have a small influence on the similarity measure. This would mean that
the "f(thetaplus) - f(thetamin)" would be entirely dominated by
the perturbation of the rotation. This is not good for the optimisation
process i think.<br><br>
Now suppose that the rotation is already close to its optimum. This means
that "f(thetaplus) - f(thetamin)" would be very small, because
the translation is only perturbed 0.1 mm. <br>
Moreover, suppose we can only obtain noisy measurements of our function
(as is assumed in the SPSA). So, instead of measuring f(theta) directly,
we measure:<br><br>
F(theta) = f(theta) + epsilon<br>
with epsilon normally distributed N(\mu, \sigma).<br><br>
'F(thetaplus) - F(thetamin)' would become extremely sensitive for noise
now. Normally we would increase c to be able to cope with high noise,
but, this would cause the whole perturbation vector to become larger. For
example, instead of 0.1 mm, we would try a perturbation of 0.5 mm, and
instead of 0.1 rad we would perturb the rotation with 0.5 rad. This is
way too much for the rotation, and would make the registration
fail.<br><br>
Groeten!<br>
Stefan.<br><br>
</body>
<br>
</html>