<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=460554812-03052006>Luis,</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=460554812-03052006></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=460554812-03052006>I
think Paul Koshevoy and Tolga Tasdizen have some recommended changes to one of
the gradient descent optimizers. I don't recall the specifics but I
believe it had something to do with checking the value of the cost function
before making a step. Perhaps they can chime in here.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=460554812-03052006></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=460554812-03052006>Jim</SPAN></FONT></DIV>
<BLOCKQUOTE>
<DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma
size=2>-----Original Message-----<SPAN class=460554812-03052006><FONT
face=Arial color=#0000ff> </FONT></SPAN><BR><B>From:</B>
insight-users-bounces+millerjv=crd.ge.com@itk.org
[mailto:insight-users-bounces+millerjv=crd.ge.com@itk.org]<B>On Behalf Of
</B>Ming Chao<BR><B>Sent:</B> Tuesday, May 02, 2006 7:32 PM<BR><B>To:</B> Luis
Ibanez<BR><B>Cc:</B> Insight Users<BR><B>Subject:</B> Re: [Insight-users]
Regular Step Gradient Descent optimizer<BR><BR></FONT></DIV>
<DIV>HI Luis,</DIV>
<DIV>Thanks for your prompt reply. Yes, I did set the optimizer (Regular Step
Gradient Descent optimizer) as the following way which I forgot to
include in last posting.</DIV>
<DIV> </DIV>
<DIV> optimizer->SetMaximumStepLength( 0.50000
);<BR> optimizer->SetMinimumStepLength( 0.01
);<BR> optimizer->SetNumberOfIterations( 200
);<BR> optimizer->SetGradientMagnitudeTolerance(
0.01*optimizer->GetGradientMagnitudeTolerance() ); </DIV>
<DIV> </DIV>
<DIV>Later on I tried to increase the step lengths as:</DIV>
<DIV> </DIV>
<DIV> optimizer->SetMaximumStepLength( 1.000 );</DIV>
<DIV> optimizer->SetMinimumStepLength( 0.05 );<BR> </DIV>
<DIV>And I had the following metric output:</DIV>
<DIV> </DIV>
<DIV>0 -0.835581 [-0.0321978, -0.466639,
0.883861]<BR>1 -0.840282 [0.399061, -0.885096,
1.68318]<BR>2 -0.837371 [0.259052, -0.883767,
2.67333]<BR>3 -0.824043 [0.75875, -0.889684,
2.68965]<BR>4 -0.829629 [0.55809, -0.881664,
2.83855]<BR>5 -0.82694 [0.679164, -0.884414,
2.80759]<BR>6 -0.828347 [0.623941, -0.882636,
2.8368]<BR>7 -0.827662 [0.566916, -0.881064,
2.86234]</DIV>
<DIV> </DIV>
<DIV>The only change is the step length but the results are similar to the
previous one. By the way for the LBFGSB optimizer I used the following
conditions:</DIV>
<DIV> </DIV>
<DIV> // (1) LBFGSB optimizer<BR>
OptimizerType::BoundSelectionType boundSelect(
transform->GetNumberOfParameters() );<BR>
OptimizerType::BoundValueType upperBound(
transform->GetNumberOfParameters() );<BR>
OptimizerType::BoundValueType lowerBound(
transform->GetNumberOfParameters() );<BR>
boundSelect.Fill( 0 );<BR> upperBound.Fill( 10.0
);<BR> lowerBound.Fill( -10.0 );<BR>
optimizer->SetBoundSelection( boundSelect ); <BR>
optimizer->SetUpperBound( upperBound );<BR>
optimizer->SetLowerBound( lowerBound );<BR>
optimizer->SetMaximumNumberOfEvaluations( 200 );<BR>
optimizer->SetMaximumNumberOfCorrections( 200 );</DIV>
<DIV> </DIV>
<DIV>Here I am not clear what you meant for best setting. I thought the
conditions I provided were reasonable. Do you see anything obviously different
which leads to different results?</DIV>
<DIV> </DIV>
<DIV>Cheers,</DIV>
<DIV>Ming</DIV>
<DIV> </DIV>
<DIV><SPAN class=gmail_quote>On 5/2/06, <B class=gmail_sendername>Luis
Ibanez</B> <<A
href="mailto:luis.ibanez@kitware.com">luis.ibanez@kitware.com</A>>
wrote:</SPAN>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid"><BR>Hi
Ming,<BR><BR><BR>Did you set the optimizer to do Minimization or
Maximization ?<BR>that is, did you used any of the following:
<BR><BR><BR> optimizer->MaximizeOn() ?<BR> optimizer->MaximizeOff() ?<BR> optimizer->MinimizeOn() ?<BR> optimizer->MinimizeOff() ?<BR><BR>How
much overlap did the images have at the end of the run <BR>with the
RegularStepNormalized Correlation ?<BR><BR>Note that the lasts iterations of
the run with<BR>RegularStepGradientDescent are advancing at very<BR>small
steps.<BR><BR>You may want to start the optimizer with a larger initial
<BR>value of the StepLength, and to change the default relaxation<BR>factor
to be 0.7 or 0.9 instead of the default value 0.5.<BR><BR>In this way, the
step length will change to 0.7 of the previous<BR>value every time that the
gradient changes directions. <BR><BR>You will get a lot of insight about the
registration process<BR>by plotting the trace of the translation in 3D. This
will<BR>show you how much the Transform is changing at every
iteration.<BR>This is not clearly conveyed just by looking at the numbers.
<BR><BR>You will find the 2D version of many of these types of plots<BR>in
the ITK Software Guide,<BR><BR> <A
href="http://www.itk.org/ItkSoftwareGuide.pdf">http://www.itk.org/ItkSoftwareGuide.pdf</A><BR><BR><BR>Note
that it is not surprising that you get different <BR>results from different
optimizers, but before you attempt<BR>to compare the results you should make
sure that you are<BR>actually using both optimizers in their best settings
and<BR>conditions. Otherwise it is just a bias and unfair
<BR>comparison.<BR><BR><BR><BR> Regards,<BR><BR><BR>
Luis<BR><BR><BR><BR>=================<BR>Ming Chao wrote:<BR>>
Hi,<BR>><BR>> When I used Regular Step Gradient Descent optimizer to
register two<BR>> images I saw an abnormal behavior. The metric value
first decreased but <BR>> after some iterations it became larger. See the
following output:<BR>><BR>> 0 -0.835581
[-0.00321978, -0.0466639, 0.0883861]<BR>> 1
-0.836504 [-0.00422488, -0.0933436, 0.176817]<BR>>
2 -0.83737 [-0.00293453, -0.139983,
0.265265]<BR>> 3 -0.838174 [0.000731481,
-0.18652, 0.353701]<BR>> 4 -0.838908 [0.00685175,
-0.232884, 0.442092]<BR>> 5 -0.839563 [0.015502,
-0.278993, 0.530404]<BR>> 6 - 0.840129
[0.0267537, -0.32476, 0.618602]<BR>> 7 -0.840598
[0.0406717, -0.370081, 0.706649]<BR>> 8 -0.84096
[ 0.0573129, -0.414844, 0.794509]<BR>> 9 -0.84121
[0.0767229, -0.458919, 0.882148]<BR>> 10 -
0.841343 [0.0989338, -0.50216, 0.969537]<BR>> 11
-0.841358 [0.123961, -0.544401, 1.05665]<BR>> 12
-0.841256 [0.151799, -0.58545 , 1.14349]<BR>> 13
-0.841041 [0.182419, -0.625084, 1.23004]<BR>> 14
- 0.840719 [0.215761, -0.663034, 1.31634]<BR>> 15
-0.840298 [0.214669, -0.680096, 1.41487]<BR>> 16
-0.839375 [0.216594, -0.696938, 1.51342]<BR>> 17
- 0.838339 [0.221686, -0.713544, 1.6119]<BR>> 18
- 0.837215 [0.230089, -0.729893, 1.7102]<BR>> 19
-0.836032 [0.241937, -0.74596, 1.80819]<BR>> 20
-0.83482 [0.257347, -0.76171, 1.90573]<BR>> 21
-0.833613 [0.276409, - 0.7771, 2.00268]<BR>> 22
-0.832445 [0.299184, -0.792076, 2.09889]<BR>> 23
-0.831349 [0.325688, -0.806571, 2.19422]<BR>> 24
-0.830357 [0.35589, -0.820502, 2.28853]<BR>> 25
-0.829494 [0.389705, -0.833768, 2.3817]<BR>> 26
-0.828783 [0.426981, -0.84624, 2.47365]<BR>> 27
-0.828236 [0.467495, -0.857749, 2.56435]<BR>> 28
-0.827859 [0.510938, -0.868055, 2.65383]<BR>> 29
-0.827633 [0.556896, -0.876774, 2.74221]<BR>> 30
-0.827525 [ 0.60475, -0.883034, 2.8298]<BR>> 31
-0.827502 [0.650812, -0.875321, 2.91822]<BR>> 32
-0.827512 [0.629235, -0.881114, 2.87349]<BR>> 33
-0.827519 [0.603546, -0.883645, 2.83067]<BR>> 34
-0.827483 [0.620407, -0.880267 , 2.84881]<BR>>
35 -0.827564 [0.608674, -0.882388, 2.84506]<BR>>
36 -0.827459 [0.613038, -0.88246, 2.84059]<BR>>
37 -0.827529 [0.610107, -0.882339, 2.84166]<BR>>
38 -0.827493 [0.611332, -0.882386,
2.8407]<BR>><BR>> However, if I change the optimizer to the LBFGSB
optimizer, I got the<BR>> following output:<BR>><BR>> 0
-0.840282 [-0.0321978, -0.466639, 0.883861]<BR>>
1 - 0.840732 [0.0493198, -0.53362, 1.01199]<BR>>
2 -0.840735 [0.0592661, -0.540774, 1.03432]<BR>>
3 -0.840751 [0.100359, -0.570829,
1.11035]<BR>><BR>> This looks reasonable. The setup for the
registration is the following:
<BR>><BR>><BR>> typedef
itk::BSplineInterpolateImageFunction<ImageType, double ><BR>>
InterpolatorType;<BR>> typedef
itk::ImageRegistrationMethod<ImageType, ImageType ><BR>>
RegistrationType; <BR>><BR>> typedef<BR>>
itk::NormalizedCorrelationImageToImageMetric<ImageType,ImageType
><BR>> MetricType;<BR>><BR>> typedef
itk::TranslationTransform< double, Dimension ><BR>> TransformType;
<BR>><BR>> Can anybody tell me why I have so different results with
different<BR>> optimizers?<BR>><BR>> Thanks,<BR>><BR>>
Ming<BR>><BR>><BR>><BR>><BR>>
------------------------------------------------------------------------
<BR>><BR>> _______________________________________________<BR>>
Insight-users mailing list<BR>> <A
href="mailto:Insight-users@itk.org">Insight-users@itk.org</A><BR>> <A
href="http://www.itk.org/mailman/listinfo/insight-users">http://www.itk.org/mailman/listinfo/insight-users</A><BR><BR><BR></BLOCKQUOTE></DIV><BR></BLOCKQUOTE></BODY></HTML>