[ITK-dev] [ITK] Optimizing composite transforms and center of transform
Bradley Lowekamp
blowekamp at mail.nih.gov
Sat Aug 2 19:48:53 EDT 2014
Hello Ali,
I created a notebook to try to demonstrate the problem you are describing:
http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/master/Importance%20of%20Center%20with%20Affine%20Registration.ipynb
If the center of an Affine transformation isn't correctly initialized then the optimization is not speedy or robust. I think it's best illustrated with these two graphs:
Notice that with the Center being the origin, all 1000 iterations are exhausted, and the metric is slowing decreasing. And this is with the translation path being around the length to 10. If the spacing was bigger or more offset from the origin the situation would be worse.
I don't think this is a bug with the registration framework. I think new initializers for composite transforms are needed, along with documentation how the 2 good ways to initialize the registration methods transforms.
Brad
On Aug 1, 2014, at 8:08 PM, Ghayoor, Ali <ali-ghayoor at uiowa.edu> wrote:
> Brad,
>
> I have exactly the same issue. I wrote a simple registration program that is run in two stages:
> -Translation
> -Affine
>
> My fixed and moving images are in 2D, and the moving image is created from the fixed image by
> rotation of 10 degree and translation of [13,17] in X and Y directions.
>
> The first stage does a translation registration and the result transform is added to a composite transform
> that is used as the initial moving transform of the second stage.
>
> After the registration the result transform of the second stage is also added to the composite transform
> to be used by the resampler.
>
> I have printed the composite transform to the screen:
>
> 2481: CompositeTransform (0x7fa359a18b00)
> 2481: RTTI typeinfo: itk::CompositeTransform<double, 2u>
> 2481: Reference Count: 2
> 2481: Modified Time: 20949
> 2481: Debug: Off
> 2481: Object Name:
> 2481: Observers:
> 2481: none
> 2481: Transforms in queue, from begin to end:
> 2481: >>>>>>>>>
> 2481: TranslationTransform (0x7fa359a199b0)
> 2481: RTTI typeinfo: itk::TranslationTransform<double, 2u>
> 2481: Reference Count: 7
> 2481: Modified Time: 1648
> 2481: Debug: Off
> 2481: Object Name:
> 2481: Observers:
> 2481: none
> 2481: Offset: [12.801, 15.8578]
> 2481: >>>>>>>>>
> 2481: AffineTransform (0x7fa359d095a0)
> 2481: RTTI typeinfo: itk::AffineTransform<double, 2u>
> 2481: Reference Count: 5
> 2481: Modified Time: 20946
> 2481: Debug: Off
> 2481: Object Name:
> 2481: Observers:
> 2481: none
> 2481: Matrix:
> 2481: 1.01593 -0.0100051
> 2481: -0.00835669 1.01113
> 2481: Offset: [-0.358829, -0.290093]
> 2481: Center: [0, 0]
> 2481: Translation: [-0.358829, -0.290093]
> 2481: Inverse:
> 2481: 0.984397 0.00974056
> 2481: 0.00813575 0.989073
> 2481: Singular: 0
> 2481: End of MultiTransform.
> 2481: <<<<<<<<<<
>
> As it can be seen, the translation transform provides a good initialization since its offset (Offset: [12.801, 15.8578]) is close to the shift parameters.
>
> However, the Affine transform fails to rotate the moving image since its Matrix is almost identity.
> Possible reason can be the improper Center of Affine transform that is fixed to [0,0], and not at geometrical center or center of mass.
>
> To make sure that I have not made any crazy mistake in my example, I simulated my example parameters in ANTs, and ANTs returned
> the same results and failed to register the images!!
>
>>>>>> ANTs command line <<<<<<<<<<<
>
> PROGPATH=/scratch/ANTS/release-new/bin
> fi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainT1SliceBorder20.png
> mi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainProtonDensitySliceR10X13Y17.png
>
> time ${PROGPATH}/antsRegistration -d 2 \
> --float \
> --output [test1, warpedMoving.nii.gz] \
> --transform "Translation[16]" \
> --metric MI[${fi},${mi},1,32,None,1] \
> --convergence [100,1e-2,5] \
> --shrink-factors 3 \
> --smoothing-sigmas 2 \
> --use-histogram-matching 1 \
> \
> --transform "Affine[1]" \
> --metric MI[${fi},${mi},1,32] \
> --convergence [100x100,1e-2,2] \
> --shrink-factors 2x1 \
> --smoothing-sigmas 1x0 \
> --use-histogram-matching 1
>
> <<<<<<<<<<<<<<<<<<<<<<<<
>
> Fixed and moving images can be loaded from the ITK data files. What do you think about this example? Is it a bug in ITK or just the parameters are not tuned finely?
>
> Thanks,
> Ali
>
> ________________________________________
> From: Johnson, Hans J
> Sent: Tuesday, July 29, 2014 6:19 PM
> To: Ghayoor, Ali
> Subject: FW: [ITK-dev] [ITK] Optimizing composite transforms and center of transform
>
> Ali,
>
> You need to follow this discussion.
>
> Hans
>
> On 7/29/14, 6:16 PM, "Bradley Lowekamp" <blowekamp at mail.nih.gov> wrote:
>
>> Hello,
>>
>> Ok, to further explore this problem, and hopeful help other people too: I
>> created an IPython Notebook viewable here:
>> http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/bl
>> ob/master/Transform%20Composition%20And%20Order.ipynb
>>
>> It doesn't come through on the static page, but I added some nifty
>> interactive slider widgets to change the rotation parameter. Which is
>> really the point to enable understanding of the parameters for the
>> transform and how they should be best optimized. The notebook should work
>> with SimpleITK >= 0.8 and IPython Notebooks 2.1. I'll encourage every one
>> to try it out :) IPython Notebooks widget are very cool!
>>
>> EXPLANATION:
>>
>> The the order of the composite transform is that the new transforms are
>> applied first.
>>
>> BUT they map from the virtual domain ( fixed ) to the moving image.
>> Therefore if your first transform moves the center of mass to the origin
>> then the second added transform's space will be centered on the mass,
>> which is good for optimizing rotation and affine parameters.
>>
>> QUESTION:
>>
>> Therefore I am inclined to conclude that its best practices to map the
>> fixed and moving image to the virtual domain such that the center of mass
>> ( or some other "center feature" ) are at the origin. This would then the
>> scale, rotation and other affine parameters around the center, with out
>> having to used the "Center" parameter the transforms currently have.
>>
>> Is this what others are doing in their registration? For general best
>> practices should this be recommend?
>>
>> @Brian I know I have seen this multi-domain description many time, but I
>> think I may have just gotten it... Is the well describe in the literature
>> some place? I think this may be an important part add the software guide.
>>
>> Thanks,
>> Brad
>>
>>
>>
>>
>> On Jul 29, 2014, at 5:38 PM, Matt McCormick <matt.mccormick at kitware.com>
>> wrote:
>>
>>> Hi Brad,
>>>
>>> Your assessment that the fixed center is important is correct.
>>>
>>> In most cases, a transform with a Center is the first transform in a
>>> Composite transform. There not any issues here. However, if the
>>> transform with a Center is further along the chain, then a
>>> CenteredTransformInitializer that was applied before the registration
>>> is started will not generate the desired result. In this case, we
>>> would have to respond to an Event in the registration process and
>>> estimated the center on the imaged after the other transforms have
>>> been applied. It is more work, but it is a more unusual case.
>>>
>>> HTH,
>>> Matt
>>>
>>>
>>>
>>> On Mon, Jul 28, 2014 at 10:11 PM, Bradley Lowekamp
>>> <blowekamp at mail.nih.gov> wrote:
>>>> Helloo Nick!
>>>>
>>>> I am glad you got back to me.
>>>>
>>>> I suspect that I have spent more time this past week looking at the
>>>> ITK affine transforms than anyone else[1] this week. With that here is
>>>> my current understanding...
>>>>
>>>> 1) The CenterTransformInitializers estimates an initial "Center" and
>>>> an initial "Translation" parameter from either the geometry or the
>>>> first moments. From my problematic experiences, getting the initial
>>>> transform which is a "good" guess is been critical for the optimizer to
>>>> head to the correct solution.
>>>>
>>>> 2) The Center parameter of a transform can either be "Fixed" or an
>>>> optimizable parameter. The transforms with the "Centered" monkier are
>>>> the the ones with the Center parameter optimizable. This issue seems to
>>>> be what you were referring to in to message.
>>>>
>>>> In many ways the point I am trying to make is independent of #2. I
>>>> have observed that it's important to have the "Center" of an affine
>>>> transform ( or sub parameterization thereof ) at the center of the
>>>> object you are trying to register. That is the coordinate space of the
>>>> optimizable parameters' point 0 is near the center. This enables
>>>> rotation to easily rotate around the center, as well as scaling to
>>>> maintain the same relative center when "zooming". This issue is
>>>> independent of wether that center point can be optimized.
>>>>
>>>> For example, consider changing the origin of an image for alignment by
>>>> say 500, and compensating with just an initial translation and not
>>>> setting the "Center" parameter. If we are trying to optimize rotation
>>>> and translation, then the optimization path would be very difficult to
>>>> traverse with these inter-dependent parameters to force it to rotate
>>>> around the center of the object. This scenario may be more common in
>>>> microscopy then medical imaging, due to microscopy frequently having
>>>> multiple subjects across large images. I could write up a IPython
>>>> Notebook to illustrate the case.
>>>>
>>>> So that is my understanding of why using a fixed center is important.
>>>> I thought this may have been shared knowledge, but perhaps is not or is
>>>> incorrect...
>>>>
>>>> Now I am trying to understand how this interacts with all the
>>>> coordinate frames involved with the ITKv4 registration framework, and
>>>> the composite transforms. Specifically the composite transform apply
>>>> the transforms in "reverse order"[2]. As I understand that that means
>>>> that the newest transform get applied first. So that transform
>>>> parameters are being optimized in the images space not in the virtual
>>>> domains we are working towards. Therefore the center of our transforms
>>>> is still important as we composite transforms?
>>>>
>>>> I hope I explained that clearly, a white board may really be needed....
>>>>
>>>> Thanks,
>>>> Brad
>>>>
>>>>
>>>> [1]
>>>> https://github.com/SimpleITK/SimpleITK/compare/master...9bc9bc8440e64d94
>>>> 099d3519eb23bcc16c7cf015
>>>> [2] http://www.itk.org/Doxygen/html/classitk_1_1CompositeTransform.html
>>>>
>>>> On Jul 28, 2014, at 8:57 PM, Nicholas Tustison <ntustison at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Brad,
>>>>>
>>>>> I apologize for the delayed response. I¹m still catching up on my
>>>>> workload after moving to CA. We might want to talk to Luis as it
>>>>> was my understanding that the ³center² component of the linear
>>>>> transforms might be a carry-over from all the ³Centered² transforms,
>>>>> e.g.,
>>>>>
>>>>>
>>>>> http://www.itk.org/Doxygen/html/classitk_1_1CenteredAffineTransform.htm
>>>>> l
>>>>>
>>>>> which, if I remember correctly, Luis said were somewhat sub optimally
>>>>> conceived but this was such a long time ago that I might be completely
>>>>> misremembering. However, fixing the center for all transforms within
>>>>> a composite transform is certainly not necessary within the specified
>>>>> framework. Rather, the matrix and offset are updated with the
>>>>> ³Center²
>>>>> being updated implicitly.
>>>>>
>>>>> Nick
>>>>>
>>>>>
>>>>>
>>>>> On Jul 24, 2014, at 7:27 AM, Bradley Lowekamp
>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> There are a lot of transforms involved in the new registration
>>>>>> framework.
>>>>>>
>>>>>> I am trying to figure out what the implications for the fix
>>>>>> parameters of the transform "Center" ( ie center of rotation/scaling
>>>>>> ) are when combined with composite transforms and the
>>>>>> fixed/moving/registration coordinate frames...
>>>>>>
>>>>>> Based on how the transforms are composed it seems necessary to set
>>>>>> the fix "Center" for subsequent transformations. Additionally, I am
>>>>>> unsure how one would "improve" ( poorly defined? ) the center for a
>>>>>> subsequent transform ie Affine after a similarity...
>>>>>>
>>>>>> Are there some documented guidance or figures to help with this
>>>>>> issue?
>>>>>> Do we have a comprehensive diagram of these transforms?
>>>>>>
>>>>>> Thanks,
>>>>>> Brad
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Kitware offers ITK Training Courses, for more information visit:
>>>> http://kitware.com/products/protraining.php
>>>>
>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://public.kitware.com/mailman/listinfo/insight-developers
>>>> _______________________________________________
>>>> Community mailing list
>>>> Community at itk.org
>>>> http://public.kitware.com/mailman/listinfo/community
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.php
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
>
> ________________________________
> Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.
> ________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140802/7e72278a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unknown.png
Type: image/png
Size: 15787 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140802/7e72278a/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unknown.png
Type: image/png
Size: 15016 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140802/7e72278a/attachment-0003.png>
More information about the Insight-developers
mailing list