[Insight-developers] Scales estimation for registration

Mon May 9 18:01:12 EDT 2011

hi bill

you have ( in a gradient descent algorithm )

for i in all_parameters ; do
  Update[i] = LearningRate * GlobalScale * ParamScales[ i ] * Gradient[ i ]
  Params_New[i] = Params_Old[i] +  Update[i]
done

where brackets index the vector terms and LearningRate * GlobalScale
is a scalar.

baohua's parameter scale estimation methods seek to estimate
ParamScales[ i ] in isolation.  so it's still up to the user to select
the  ( LearningRate * GlobalScale )  term.

estimating that term requires knowledge of the gradient.

the problem with the test data is that Gradient[i] for the rotation
term appears to be far from zero, even though there is no rotation in
the ground truth.   the test is well-designed to break a parameter
estimation algorithm.  that is, there is an affine transform operating
on images that only have a translation between them.

so, the point is, if one knows there may be a translation alone, then
one should use a translation transform first, then follow that with an
affine transform.    that is the usual best-practice approach.  if one
were to apply this approach in this case, with the parameter scale
estimator, i'd guess you'd get reasonable results.   but you'd still
have to manually set   ( LearningRate * GlobalScale ) ... this
limitation be overcome as well (basically as stefan described) and
Baohua is working on it.

brian

On Mon, May 9, 2011 at 5:31 PM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>
> My concern is the estimation of the translation scales. It seems the global
> scale factor is used to scale the "estimated scales". I guess, indirectly,
> it interacts with the learning rate. So it is also of concern.
>
>
> On Mon, May 9, 2011 at 1:42 PM, Baohua Wu <baowu99 at gmail.com> wrote:
>>
>> hi brian,
>>
>> thank you for the information.
>>
>> to my knowledge, the test was passed with the synthetic data, and the
>> scales worked well with several real images. i think bill's conern was
>> mainly about how to set the global factor (or learning rate).
>>
>> we have been trying to estimate this global factor as well. some
>> strategies include the norm of gradient, the parameter space, etc.
>>
>> baohua
>>
>> ----- Original Message ----- From: "brian avants" <stnava at gmail.com>
>> To: "Baohua Wu" <baowu99 at gmail.com>; "Nicholas Tustison"
>> <ntustison at gmail.com>
>> Cc: "Stefan Klein" <s.klein at erasmusmc.nl>; "Bill Lorensen"
>> <bill.lorensen at gmail.com>; "Marius Staring (registration)"
>> <M.Staring at lumc.nl>; "Stefan Klein" <stefan at isi.uu.nl>; "Insight Developers"
>> <insight-developers at itk.org>
>> Sent: Monday, May 09, 2011 10:05 AM
>> Subject: Re: [Insight-developers] Scales estimation for registration
>>
>>
>> hi baohua
>>
>> nick may have some code that is relevant.
>>
>> also, you may want to let people know about your other strategies for
>> estimating the parameter scales.  for instance,  the methods that take
>> into account the norm of the gradient itself, as well as as that build
>> estimates of the parameter space.
>>
>> brian
>>
>>
>>
>>
>> On Mon, May 9, 2011 at 9:24 AM, Baohua Wu <baowu99 at gmail.com> wrote:
>>>
>>> Stefan,
>>>
>>> Thank you for the advice. I am glad to know your method works for scales
>>> and learning rate. I will try it .
>>>
>>> Baohua
>>>
>>> ----- Original Message ----- From: "Stefan Klein" <s.klein at erasmusmc.nl>
>>> To: "Bill Lorensen" <bill.lorensen at gmail.com>
>>> Cc: "Marius Staring (registration)" <M.Staring at lumc.nl>;
>>> <baowu99 at gmail.com>; "Stefan Klein" <stefan at isi.uu.nl>; "Insight
>>> Developers"
>>> <insight-developers at itk.org>
>>> Sent: Monday, May 09, 2011 7:43 AM
>>> Subject: Re: [Insight-developers] Scales estimation for registration
>>>
>>>
>>>> Hi Bill,
>>>>
>>>> The method in elastix works well in my experience.
>>>>
>>>> The method is explained here:
>>>> http://dx.doi.org/10.1007/s11263-008-0168-y
>>>>
>>>> Equation 61/62
>>>> But that final expression should be squared because of the definition of
>>>> the scales in ITK.
>>>>
>>>> The method needs to know the fixed image domain and the transform. So it
>>>> would fit in the same framework as Bao's class. The methods declared at
>>>> lines 147 to 161 should probably be protected by the way?
>>>>
>>>> Just to clarify: the AutomaticScalesEstimation in elastix only computes
>>>> the scales, not the global LearningRate. I guess the GlobalScalingFactor
>>>> serves to determine the global LearningRate?
>>>>
>>>> To automatically determine an optimal global learning rate, you would
>>>> need
>>>> the CostFunction as well. (since multiplying the costfunction by 10
>>>> would
>>>> cause the optimal learning rate to be divided by 10, for example).
>>>>
>>>> Good idea to make this a separate class!
>>>> Kind regards,
>>>> Stefan
>>>>
>>>>
>>>>
>>>> On 06/05/2011 22:52, Bill Lorensen wrote:
>>>>>
>>>>> Marius, Stefan and Bao,
>>>>>
>>>>> One of the major weaknesses in ITK's registration framework is the
>>>>> estimation of parameters. In particular, the estimation of the scales
>>>>> seems to require special magic. My experience has been that the
>>>>> selection of a proper scaling for translation is particularly
>>>>> difficult. ITK traditionally (and currently) leaves the scaling
>>>>> decision up to the application. There are currently two possible
>>>>> approaches to this estimation.
>>>>>
>>>>> 1) Bao's itkParameterScaleEstimator proposed in this gerrit topic:
>>>>> http://review.source.kitware.com/#change,1499
>>>>> 2) Elastix's (http://elastix.isi.uu.nl/) AutomaticScalesEstimation
>>>>> method implementation in elxTransformBase.
>>>>>
>>>>> For Marius and Stefan I have questions:
>>>>>
>>>>> a) How successful is elastix's AutomaticScalesEstimation?
>>>>> b) Currently this is implemented as a method in
>>>>> elxTransformBase. Could it be implemented in an API similar to Bao's
>>>>> itkParameterScaleEstimator?
>>>>>
>>>>>
>>>>> For Bao's implementation, my concern is the addition of a free
>>>>> parameter GlobalScalingFactor. It is not clear how to pick a proper
>>>>> value. To me there is as much mystery in this parameter as there is in
>>>>> picking a proper translation scaling
>>>>>
>>>>> Bill
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Powered by www.kitware.com
>>>>>
>>>>> Visit other Kitware open-source projects at
>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>
>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>> http://kitware.com/products/protraining.html
>>>>>
>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>
>>>>> Follow this link to subscribe/unsubscribe:
>>>>> http://www.itk.org/mailman/listinfo/insight-developers
>>>>
>>>> --
>>>> Stefan Klein
>>>> +31 10 7043049
>>>> http://www.bigr.nl/people/StefanKlein
>>>>
>>>
>>>
>>
>
>