[Insight-developers] Scales estimation for registration

[Stefan Klein]

Hi Bill, thanks, I now just see the previous mails on the list on this 
topic where Bao already explained that it indeed serves to estimate the 
learning rate.

In the article that I just mentioned I also try to guess a reasonable 
learning rate. That code is also implemented in elastix, in a different 
place. The article focuses on stochastic gradient descent, whereas the 
gradient descent method in ITK is meant as a deterministic optimizer. 
Equation 47 in the paper gives the result for the deterministic case. 
a_MAX is essentially the 'optimal' LearningRate for a deterministic 
gradient descent optimizer. The parameter delta is still user-defined 
but does have an intuitive meaning; setting it to the size of a voxel 
usually works fine.

For stochastic optimizers, there are some additional steps needed, which 
is summarised in equation 49 (factor \eta).

To calculate the terms in equation 47, the cost function's 
GetDerivative() is needed. So for that, the interface of the 
itkParameterScaleEstimator should be extended.

Kind regards,
Stefan


On 09/05/2011 13:48, Bill Lorensen wrote:
> Stefan,
>
> I think the GlobalScalingFactor is a value to normalize the computed
> scales. I'll take a look at your method.
>
> Thanks,
>
> Bill
>
> On Mon, May 9, 2011 at 7:43 AM, Stefan Klein <s.klein at erasmusmc.nl
> <mailto:s.klein at erasmusmc.nl>> wrote:
>
>     Hi Bill,
>
>     The method in elastix works well in my experience.
>
>     The method is explained here:
>     http://dx.doi.org/10.1007/s11263-008-0168-y
>
>     Equation 61/62
>     But that final expression should be squared because of the
>     definition of the scales in ITK.
>
>     The method needs to know the fixed image domain and the transform.
>     So it would fit in the same framework as Bao's class. The methods
>     declared at lines 147 to 161 should probably be protected by the way?
>
>     Just to clarify: the AutomaticScalesEstimation in elastix only
>     computes the scales, not the global LearningRate. I guess the
>     GlobalScalingFactor serves to determine the global LearningRate?
>
>     To automatically determine an optimal global learning rate, you
>     would need the CostFunction as well. (since multiplying the
>     costfunction by 10 would cause the optimal learning rate to be
>     divided by 10, for example).
>
>     Good idea to make this a separate class!
>     Kind regards,
>     Stefan
>
>
>
>
>     On 06/05/2011 22:52, Bill Lorensen wrote:
>
>         Marius, Stefan and Bao,
>
>         One of the major weaknesses in ITK's registration framework is the
>         estimation of parameters. In particular, the estimation of the
>         scales
>         seems to require special magic. My experience has been that the
>         selection of a proper scaling for translation is particularly
>         difficult. ITK traditionally (and currently) leaves the scaling
>         decision up to the application. There are currently two possible
>         approaches to this estimation.
>
>         1) Bao's itkParameterScaleEstimator proposed in this gerrit topic:
>         http://review.source.kitware.com/#change,1499
>         2) Elastix's (http://elastix.isi.uu.nl/) AutomaticScalesEstimation
>             method implementation in elxTransformBase.
>
>         For Marius and Stefan I have questions:
>
>         a) How successful is elastix's AutomaticScalesEstimation?
>         b) Currently this is implemented as a method in
>             elxTransformBase. Could it be implemented in an API similar
>         to Bao's
>             itkParameterScaleEstimator?
>
>
>         For Bao's implementation, my concern is the addition of a free
>         parameter GlobalScalingFactor. It is not clear how to pick a proper
>         value. To me there is as much mystery in this parameter as there
>         is in
>         picking a proper translation scaling
>
>         Bill
>
>
>
>
>
>         _______________________________________________
>         Powered by www.kitware.com <http://www.kitware.com>
>
>         Visit other Kitware open-source projects at
>         http://www.kitware.com/opensource/opensource.html
>
>         Kitware offers ITK Training Courses, for more information visit:
>         http://kitware.com/products/protraining.html
>
>         Please keep messages on-topic and check the ITK FAQ at:
>         http://www.itk.org/Wiki/ITK_FAQ
>
>         Follow this link to subscribe/unsubscribe:
>         http://www.itk.org/mailman/listinfo/insight-developers
>
>
>     --
>     Stefan Klein
>     +31 10 7043049 <tel:%2B31%2010%207043049>
>     http://www.bigr.nl/people/StefanKlein
>
>

-- 
Stefan Klein
+31 10 7043049
http://www.bigr.nl/people/StefanKlein