[ITK-dev] [ITK] Optimizing composite transforms and center of transform
Bradley Lowekamp
blowekamp at mail.nih.gov
Wed Aug 13 12:23:18 EDT 2014
Nick,
Thanks you for working on the patch. It appears that I was incorrect with my proposed formula for propagating the center.
Hans, Ali, and myself have continued this discussion over the the last couple of days. After further experimentation with IPython notebooks and composite transform we developed a better test cases to interact with. This lead to the conclusion that the Center parameter is not an point in the moving image's domain, but a point in the fixed. Therefore the same center can be used for all composite affine transforms and it can more simply be just copied along. Here is the corrected statement:
For a Composite transformed defined as ϕ=T0(T1(x)) where ϕ:Ωfixed→Ωmoving then if C such that C∈Ωfixed is the correct center initialization for T0, then the correct center initialization for T1 is still C, as it is set in the in the domain ( not the range ) of the individual transforms.
And here is a link to a viewer of the revisited notebooks:
http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/UpdateCompositeTransformCenter/Transform%20Composition%20And%20Order.ipynb
Brad
On Aug 5, 2014, at 8:03 AM, Bradley Lowekamp <blowekamp at mail.nih.gov> wrote:
> Ali,
>
> Glad to hear the registration iteration graphs where effective.
>
> Yes, you are right you can initialize the parameters of the second stage affine registration. Specifically the Center. However, it not readily apparent ( no example, no filter/initializer ) how that center fixed parameter should be initialized. And the "CenteredTransformInitializer" is not the correct way.
>
> For a Composite transformed defined as ϕ=T0(T1(x)) where ϕ:Ωfixed→Ωmoving then if C such that C∈Ωmoving is the correct center initialization for T1, then the correct center initialization for T0 when in the composite transform is C′=T−10(C).
>
> Cool pasting from the IPython Notebook with inline latex appears to have worked :)
>
> While it is easy to do this with the MomentsCaluclator and manually doing the transform inverse, I think to make it more readily apparent and easier to use it should be available as a filter/initializer.
>
> There are several things that are done manually in the examples that I would think should be updated to use the latest filters and initializers. For example The BSplines are manually initialized instead of using the BSplineTransform Initializer[1]. Additionally, conversion of BSpline to a deformation field[2], and the the upsampling of the BSpine[3], though that last one still looks like ITKv3 framework, so maybe it'll be updated to the adaptors with v4. However I am after simple functions for these common tasks for SimpleITK, so I have a bit different perspective then one who is OK with implementing a 50 line solution in C++ and being OK with the interface.
>
> We'll discuss these initialization issue further this afternoon.
>
> Brad
>
> [1] https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration4.cxx#L215-L235
> [2] https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration4.cxx#L401-L430
> [3] https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration6.cxx#L273-L330
>
> On Aug 4, 2014, at 11:56 PM, Ghayoor, Ali <ali-ghayoor at uiowa.edu> wrote:
>
>> Hello Brad,
>>
>> It is an awesome way for demonstration, and thanks for verification of my example. It seems that this email is sent on Saturday, but unfortunately I got that this morning! And I should inspect the reason!
>> Anyway, over the weekend I ran almost the same experiment in ITK and got close results to you.
>>
>> Also, today I tried to make the registration process work when Affine Transform is used in a multi stage structure, and here is my suggestion:
>>
>> I think we do not need to bother ourselves to think about a new optimizer or a new initializer for the composite transform since we can initialize the fixed parameters of each transform individually at the beginning of each stage.
>>
>> Remember how we run a BSpline registration stage after some linear registrations. At the beginning of the BSpline stage, we should instantiate a BSpline transform object and define its fixed parameters (grid size, etc). Then, we pass this initialized transform to the registration filter using “SetInitialTransform()”, yet we can initialize this stage by passing a composite transform, resultant from the previous stages, to the registration filter by “SetMovingInitialTransform()”.
>>
>> The same condition holds for all other transform types with fixed unoptimizable parameters like the center of rotation in our Affine transform case. Attachment file includes a multi stage registration example that I have created for the new software guide. In this example, we use a simple 2 stages registration process for a multi modal problem that moving image is misaligned from the fixed image by a translation shift and rotation. In registration process, a translation transform is followed by an affine transform. Also, an initial transform is used at the beginning.
>>
>> If I do not set the center of the affine transform, the registration fails as you have shown in your Ipython example. However, if at the beginning of the second stage, I compute the geometrical center (or center of the mass) of the fixed image, and pass that to the affine transform using “SetCenter()” or “SetFixedParameters()”, the registration succeeds. Please take a quick look in the attached file that also includes lots of explanations.
>>
>> Therefore, we can follow the same procedure for all other transform types that their fixed parameters are important in the optimization, but those fixed parameters cannot be set explicitly inside a composite transform. Please let me know with you think about this.
>>
>> Thank you,
>> Ali
>>
>>
>> From: Bradley Lowekamp <blowekamp at mail.nih.gov>
>> Date: Saturday, August 2, 2014 at 6:48 PM
>> To: Ali Ghayoor <ali-ghayoor at uiowa.edu>
>> Cc: Hans Johnson <hans-johnson at uiowa.edu>, Insight Developers List <insight-developers at public.kitware.com>
>> Subject: Re: [ITK-dev] [ITK] Optimizing composite transforms and center of transform
>>
>> Hello Ali,
>>
>> I created a notebook to try to demonstrate the problem you are describing:
>> http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/master/Importance%20of%20Center%20with%20Affine%20Registration.ipynb
>>
>> If the center of an Affine transformation isn't correctly initialized then the optimization is not speedy or robust. I think it's best illustrated with these two graphs:
>>
>> <unknown.png>
>> <unknown.png>
>>
>> Notice that with the Center being the origin, all 1000 iterations are exhausted, and the metric is slowing decreasing. And this is with the translation path being around the length to 10. If the spacing was bigger or more offset from the origin the situation would be worse.
>>
>> I don't think this is a bug with the registration framework. I think new initializers for composite transforms are needed, along with documentation how the 2 good ways to initialize the registration methods transforms.
>>
>>
>> Brad
>>
>> On Aug 1, 2014, at 8:08 PM, Ghayoor, Ali <ali-ghayoor at uiowa.edu> wrote:
>>
>>> Brad,
>>>
>>> I have exactly the same issue. I wrote a simple registration program that is run in two stages:
>>> -Translation
>>> -Affine
>>>
>>> My fixed and moving images are in 2D, and the moving image is created from the fixed image by
>>> rotation of 10 degree and translation of [13,17] in X and Y directions.
>>>
>>> The first stage does a translation registration and the result transform is added to a composite transform
>>> that is used as the initial moving transform of the second stage.
>>>
>>> After the registration the result transform of the second stage is also added to the composite transform
>>> to be used by the resampler.
>>>
>>> I have printed the composite transform to the screen:
>>>
>>> 2481: CompositeTransform (0x7fa359a18b00)
>>> 2481: RTTI typeinfo: itk::CompositeTransform<double, 2u>
>>> 2481: Reference Count: 2
>>> 2481: Modified Time: 20949
>>> 2481: Debug: Off
>>> 2481: Object Name:
>>> 2481: Observers:
>>> 2481: none
>>> 2481: Transforms in queue, from begin to end:
>>> 2481: >>>>>>>>>
>>> 2481: TranslationTransform (0x7fa359a199b0)
>>> 2481: RTTI typeinfo: itk::TranslationTransform<double, 2u>
>>> 2481: Reference Count: 7
>>> 2481: Modified Time: 1648
>>> 2481: Debug: Off
>>> 2481: Object Name:
>>> 2481: Observers:
>>> 2481: none
>>> 2481: Offset: [12.801, 15.8578]
>>> 2481: >>>>>>>>>
>>> 2481: AffineTransform (0x7fa359d095a0)
>>> 2481: RTTI typeinfo: itk::AffineTransform<double, 2u>
>>> 2481: Reference Count: 5
>>> 2481: Modified Time: 20946
>>> 2481: Debug: Off
>>> 2481: Object Name:
>>> 2481: Observers:
>>> 2481: none
>>> 2481: Matrix:
>>> 2481: 1.01593 -0.0100051
>>> 2481: -0.00835669 1.01113
>>> 2481: Offset: [-0.358829, -0.290093]
>>> 2481: Center: [0, 0]
>>> 2481: Translation: [-0.358829, -0.290093]
>>> 2481: Inverse:
>>> 2481: 0.984397 0.00974056
>>> 2481: 0.00813575 0.989073
>>> 2481: Singular: 0
>>> 2481: End of MultiTransform.
>>> 2481: <<<<<<<<<<
>>>
>>> As it can be seen, the translation transform provides a good initialization since its offset (Offset: [12.801, 15.8578]) is close to the shift parameters.
>>>
>>> However, the Affine transform fails to rotate the moving image since its Matrix is almost identity.
>>> Possible reason can be the improper Center of Affine transform that is fixed to [0,0], and not at geometrical center or center of mass.
>>>
>>> To make sure that I have not made any crazy mistake in my example, I simulated my example parameters in ANTs, and ANTs returned
>>> the same results and failed to register the images!!
>>>
>>>>>>>> ANTs command line <<<<<<<<<<<
>>>
>>> PROGPATH=/scratch/ANTS/release-new/bin
>>> fi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainT1SliceBorder20.png
>>> mi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainProtonDensitySliceR10X13Y17.png
>>>
>>> time ${PROGPATH}/antsRegistration -d 2 \
>>> --float \
>>> --output [test1, warpedMoving.nii.gz] \
>>> --transform "Translation[16]" \
>>> --metric MI[${fi},${mi},1,32,None,1] \
>>> --convergence [100,1e-2,5] \
>>> --shrink-factors 3 \
>>> --smoothing-sigmas 2 \
>>> --use-histogram-matching 1 \
>>> \
>>> --transform "Affine[1]" \
>>> --metric MI[${fi},${mi},1,32] \
>>> --convergence [100x100,1e-2,2] \
>>> --shrink-factors 2x1 \
>>> --smoothing-sigmas 1x0 \
>>> --use-histogram-matching 1
>>>
>>> <<<<<<<<<<<<<<<<<<<<<<<<
>>>
>>> Fixed and moving images can be loaded from the ITK data files. What do you think about this example? Is it a bug in ITK or just the parameters are not tuned finely?
>>>
>>> Thanks,
>>> Ali
>>>
>>> ________________________________________
>>> From: Johnson, Hans J
>>> Sent: Tuesday, July 29, 2014 6:19 PM
>>> To: Ghayoor, Ali
>>> Subject: FW: [ITK-dev] [ITK] Optimizing composite transforms and center of transform
>>>
>>> Ali,
>>>
>>> You need to follow this discussion.
>>>
>>> Hans
>>>
>>> On 7/29/14, 6:16 PM, "Bradley Lowekamp" <blowekamp at mail.nih.gov> wrote:
>>>
>>>> Hello,
>>>>
>>>> Ok, to further explore this problem, and hopeful help other people too: I
>>>> created an IPython Notebook viewable here:
>>>> http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/bl
>>>> ob/master/Transform%20Composition%20And%20Order.ipynb
>>>>
>>>> It doesn't come through on the static page, but I added some nifty
>>>> interactive slider widgets to change the rotation parameter. Which is
>>>> really the point to enable understanding of the parameters for the
>>>> transform and how they should be best optimized. The notebook should work
>>>> with SimpleITK >= 0.8 and IPython Notebooks 2.1. I'll encourage every one
>>>> to try it out :) IPython Notebooks widget are very cool!
>>>>
>>>> EXPLANATION:
>>>>
>>>> The the order of the composite transform is that the new transforms are
>>>> applied first.
>>>>
>>>> BUT they map from the virtual domain ( fixed ) to the moving image.
>>>> Therefore if your first transform moves the center of mass to the origin
>>>> then the second added transform's space will be centered on the mass,
>>>> which is good for optimizing rotation and affine parameters.
>>>>
>>>> QUESTION:
>>>>
>>>> Therefore I am inclined to conclude that its best practices to map the
>>>> fixed and moving image to the virtual domain such that the center of mass
>>>> ( or some other "center feature" ) are at the origin. This would then the
>>>> scale, rotation and other affine parameters around the center, with out
>>>> having to used the "Center" parameter the transforms currently have.
>>>>
>>>> Is this what others are doing in their registration? For general best
>>>> practices should this be recommend?
>>>>
>>>> @Brian I know I have seen this multi-domain description many time, but I
>>>> think I may have just gotten it... Is the well describe in the literature
>>>> some place? I think this may be an important part add the software guide.
>>>>
>>>> Thanks,
>>>> Brad
>>>>
>>>>
>>>>
>>>>
>>>> On Jul 29, 2014, at 5:38 PM, Matt McCormick <matt.mccormick at kitware.com>
>>>> wrote:
>>>>
>>>>> Hi Brad,
>>>>>
>>>>> Your assessment that the fixed center is important is correct.
>>>>>
>>>>> In most cases, a transform with a Center is the first transform in a
>>>>> Composite transform. There not any issues here. However, if the
>>>>> transform with a Center is further along the chain, then a
>>>>> CenteredTransformInitializer that was applied before the registration
>>>>> is started will not generate the desired result. In this case, we
>>>>> would have to respond to an Event in the registration process and
>>>>> estimated the center on the imaged after the other transforms have
>>>>> been applied. It is more work, but it is a more unusual case.
>>>>>
>>>>> HTH,
>>>>> Matt
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jul 28, 2014 at 10:11 PM, Bradley Lowekamp
>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>> Helloo Nick!
>>>>>>
>>>>>> I am glad you got back to me.
>>>>>>
>>>>>> I suspect that I have spent more time this past week looking at the
>>>>>> ITK affine transforms than anyone else[1] this week. With that here is
>>>>>> my current understanding...
>>>>>>
>>>>>> 1) The CenterTransformInitializers estimates an initial "Center" and
>>>>>> an initial "Translation" parameter from either the geometry or the
>>>>>> first moments. From my problematic experiences, getting the initial
>>>>>> transform which is a "good" guess is been critical for the optimizer to
>>>>>> head to the correct solution.
>>>>>>
>>>>>> 2) The Center parameter of a transform can either be "Fixed" or an
>>>>>> optimizable parameter. The transforms with the "Centered" monkier are
>>>>>> the the ones with the Center parameter optimizable. This issue seems to
>>>>>> be what you were referring to in to message.
>>>>>>
>>>>>> In many ways the point I am trying to make is independent of #2. I
>>>>>> have observed that it's important to have the "Center" of an affine
>>>>>> transform ( or sub parameterization thereof ) at the center of the
>>>>>> object you are trying to register. That is the coordinate space of the
>>>>>> optimizable parameters' point 0 is near the center. This enables
>>>>>> rotation to easily rotate around the center, as well as scaling to
>>>>>> maintain the same relative center when "zooming". This issue is
>>>>>> independent of wether that center point can be optimized.
>>>>>>
>>>>>> For example, consider changing the origin of an image for alignment by
>>>>>> say 500, and compensating with just an initial translation and not
>>>>>> setting the "Center" parameter. If we are trying to optimize rotation
>>>>>> and translation, then the optimization path would be very difficult to
>>>>>> traverse with these inter-dependent parameters to force it to rotate
>>>>>> around the center of the object. This scenario may be more common in
>>>>>> microscopy then medical imaging, due to microscopy frequently having
>>>>>> multiple subjects across large images. I could write up a IPython
>>>>>> Notebook to illustrate the case.
>>>>>>
>>>>>> So that is my understanding of why using a fixed center is important.
>>>>>> I thought this may have been shared knowledge, but perhaps is not or is
>>>>>> incorrect...
>>>>>>
>>>>>> Now I am trying to understand how this interacts with all the
>>>>>> coordinate frames involved with the ITKv4 registration framework, and
>>>>>> the composite transforms. Specifically the composite transform apply
>>>>>> the transforms in "reverse order"[2]. As I understand that that means
>>>>>> that the newest transform get applied first. So that transform
>>>>>> parameters are being optimized in the images space not in the virtual
>>>>>> domains we are working towards. Therefore the center of our transforms
>>>>>> is still important as we composite transforms?
>>>>>>
>>>>>> I hope I explained that clearly, a white board may really be needed....
>>>>>>
>>>>>> Thanks,
>>>>>> Brad
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> https://github.com/SimpleITK/SimpleITK/compare/master...9bc9bc8440e64d94
>>>>>> 099d3519eb23bcc16c7cf015
>>>>>> [2] http://www.itk.org/Doxygen/html/classitk_1_1CompositeTransform.html
>>>>>>
>>>>>> On Jul 28, 2014, at 8:57 PM, Nicholas Tustison <ntustison at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Brad,
>>>>>>>
>>>>>>> I apologize for the delayed response. I¹m still catching up on my
>>>>>>> workload after moving to CA. We might want to talk to Luis as it
>>>>>>> was my understanding that the ³center² component of the linear
>>>>>>> transforms might be a carry-over from all the ³Centered² transforms,
>>>>>>> e.g.,
>>>>>>>
>>>>>>>
>>>>>>> http://www.itk.org/Doxygen/html/classitk_1_1CenteredAffineTransform.htm
>>>>>>> l
>>>>>>>
>>>>>>> which, if I remember correctly, Luis said were somewhat sub optimally
>>>>>>> conceived but this was such a long time ago that I might be completely
>>>>>>> misremembering. However, fixing the center for all transforms within
>>>>>>> a composite transform is certainly not necessary within the specified
>>>>>>> framework. Rather, the matrix and offset are updated with the
>>>>>>> ³Center²
>>>>>>> being updated implicitly.
>>>>>>>
>>>>>>> Nick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Jul 24, 2014, at 7:27 AM, Bradley Lowekamp
>>>>>>> <blowekamp at mail.nih.gov> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> There are a lot of transforms involved in the new registration
>>>>>>>> framework.
>>>>>>>>
>>>>>>>> I am trying to figure out what the implications for the fix
>>>>>>>> parameters of the transform "Center" ( ie center of rotation/scaling
>>>>>>>> ) are when combined with composite transforms and the
>>>>>>>> fixed/moving/registration coordinate frames...
>>>>>>>>
>>>>>>>> Based on how the transforms are composed it seems necessary to set
>>>>>>>> the fix "Center" for subsequent transformations. Additionally, I am
>>>>>>>> unsure how one would "improve" ( poorly defined? ) the center for a
>>>>>>>> subsequent transform ie Affine after a similarity...
>>>>>>>>
>>>>>>>> Are there some documented guidance or figures to help with this
>>>>>>>> issue?
>>>>>>>> Do we have a comprehensive diagram of these transforms?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Brad
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at
>>>>>> http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Kitware offers ITK Training Courses, for more information visit:
>>>>>> http://kitware.com/products/protraining.php
>>>>>>
>>>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://public.kitware.com/mailman/listinfo/insight-developers
>>>>>> _______________________________________________
>>>>>> Community mailing list
>>>>>> Community at itk.org
>>>>>> http://public.kitware.com/mailman/listinfo/community
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Kitware offers ITK Training Courses, for more information visit:
>>>> http://kitware.com/products/protraining.php
>>>>
>>>> Please keep messages on-topic and check the ITK FAQ at:
>>>> http://www.itk.org/Wiki/ITK_FAQ
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://public.kitware.com/mailman/listinfo/insight-developers
>>>
>>>
>>>
>>> ________________________________
>>> Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.
>>> ________________________________
>>
>> <unknown.png><unknown.png><MultiStageImageRegistration1.cxx>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
> _______________________________________________
> Community mailing list
> Community at itk.org
> http://public.kitware.com/mailman/listinfo/community
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140813/a26b276c/attachment-0001.html>
More information about the Insight-developers
mailing list