[ITK-dev] [ITK] Optimizing composite transforms and center of transform
Ghayoor, Ali
ali-ghayoor at uiowa.edu
Mon Aug 4 23:56:52 EDT 2014
Hello Brad,
It is an awesome way for demonstration, and thanks for verification of my example. It seems that this email is sent on Saturday, but unfortunately I got that this morning! And I should inspect the reason!
Anyway, over the weekend I ran almost the same experiment in ITK and got close results to you.
Also, today I tried to make the registration process work when Affine Transform is used in a multi stage structure, and here is my suggestion:
I think we do not need to bother ourselves to think about a new optimizer or a new initializer for the composite transform since we can initialize the fixed parameters of each transform individually at the beginning of each stage.
Remember how we run a BSpline registration stage after some linear registrations. At the beginning of the BSpline stage, we should instantiate a BSpline transform object and define its fixed parameters (grid size, etc). Then, we pass this initialized transform to the registration filter using “SetInitialTransform()”, yet we can initialize this stage by passing a composite transform, resultant from the previous stages, to the registration filter by “SetMovingInitialTransform()”.
The same condition holds for all other transform types with fixed unoptimizable parameters like the center of rotation in our Affine transform case. Attachment file includes a multi stage registration example that I have created for the new software guide. In this example, we use a simple 2 stages registration process for a multi modal problem that moving image is misaligned from the fixed image by a translation shift and rotation. In registration process, a translation transform is followed by an affine transform. Also, an initial transform is used at the beginning.
If I do not set the center of the affine transform, the registration fails as you have shown in your Ipython example. However, if at the beginning of the second stage, I compute the geometrical center (or center of the mass) of the fixed image, and pass that to the affine transform using “SetCenter()” or “SetFixedParameters()”, the registration succeeds. Please take a quick look in the attached file that also includes lots of explanations.
Therefore, we can follow the same procedure for all other transform types that their fixed parameters are important in the optimization, but those fixed parameters cannot be set explicitly inside a composite transform. Please let me know with you think about this.
Thank you,
Ali
From: Bradley Lowekamp <blowekamp at mail.nih.gov<mailto:blowekamp at mail.nih.gov>>
Date: Saturday, August 2, 2014 at 6:48 PM
To: Ali Ghayoor <ali-ghayoor at uiowa.edu<mailto:ali-ghayoor at uiowa.edu>>
Cc: Hans Johnson <hans-johnson at uiowa.edu<mailto:hans-johnson at uiowa.edu>>, Insight Developers List <insight-developers at public.kitware.com<mailto:insight-developers at public.kitware.com>>
Subject: Re: [ITK-dev] [ITK] Optimizing composite transforms and center of transform
Hello Ali,
I created a notebook to try to demonstrate the problem you are describing:
http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/master/Importance%20of%20Center%20with%20Affine%20Registration.ipynb
If the center of an Affine transformation isn't correctly initialized then the optimization is not speedy or robust. I think it's best illustrated with these two graphs:
[unknown.png]
[unknown.png]
Notice that with the Center being the origin, all 1000 iterations are exhausted, and the metric is slowing decreasing. And this is with the translation path being around the length to 10. If the spacing was bigger or more offset from the origin the situation would be worse.
I don't think this is a bug with the registration framework. I think new initializers for composite transforms are needed, along with documentation how the 2 good ways to initialize the registration methods transforms.
Brad
On Aug 1, 2014, at 8:08 PM, Ghayoor, Ali <ali-ghayoor at uiowa.edu<mailto:ali-ghayoor at uiowa.edu>> wrote:
Brad,
I have exactly the same issue. I wrote a simple registration program that is run in two stages:
-Translation
-Affine
My fixed and moving images are in 2D, and the moving image is created from the fixed image by
rotation of 10 degree and translation of [13,17] in X and Y directions.
The first stage does a translation registration and the result transform is added to a composite transform
that is used as the initial moving transform of the second stage.
After the registration the result transform of the second stage is also added to the composite transform
to be used by the resampler.
I have printed the composite transform to the screen:
2481: CompositeTransform (0x7fa359a18b00)
2481: RTTI typeinfo: itk::CompositeTransform<double, 2u>
2481: Reference Count: 2
2481: Modified Time: 20949
2481: Debug: Off
2481: Object Name:
2481: Observers:
2481: none
2481: Transforms in queue, from begin to end:
2481: >>>>>>>>>
2481: TranslationTransform (0x7fa359a199b0)
2481: RTTI typeinfo: itk::TranslationTransform<double, 2u>
2481: Reference Count: 7
2481: Modified Time: 1648
2481: Debug: Off
2481: Object Name:
2481: Observers:
2481: none
2481: Offset: [12.801, 15.8578]
2481: >>>>>>>>>
2481: AffineTransform (0x7fa359d095a0)
2481: RTTI typeinfo: itk::AffineTransform<double, 2u>
2481: Reference Count: 5
2481: Modified Time: 20946
2481: Debug: Off
2481: Object Name:
2481: Observers:
2481: none
2481: Matrix:
2481: 1.01593 -0.0100051
2481: -0.00835669 1.01113
2481: Offset: [-0.358829, -0.290093]
2481: Center: [0, 0]
2481: Translation: [-0.358829, -0.290093]
2481: Inverse:
2481: 0.984397 0.00974056
2481: 0.00813575 0.989073
2481: Singular: 0
2481: End of MultiTransform.
2481: <<<<<<<<<<
As it can be seen, the translation transform provides a good initialization since its offset (Offset: [12.801, 15.8578]) is close to the shift parameters.
However, the Affine transform fails to rotate the moving image since its Matrix is almost identity.
Possible reason can be the improper Center of Affine transform that is fixed to [0,0], and not at geometrical center or center of mass.
To make sure that I have not made any crazy mistake in my example, I simulated my example parameters in ANTs, and ANTs returned
the same results and failed to register the images!!
ANTs command line <<<<<<<<<<<
PROGPATH=/scratch/ANTS/release-new/bin
fi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainT1SliceBorder20.png
mi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainProtonDensitySliceR10X13Y17.png
time ${PROGPATH}/antsRegistration -d 2 \
--float \
--output [test1, warpedMoving.nii.gz] \
--transform "Translation[16]" \
--metric MI[${fi},${mi},1,32,None,1] \
--convergence [100,1e-2,5] \
--shrink-factors 3 \
--smoothing-sigmas 2 \
--use-histogram-matching 1 \
\
--transform "Affine[1]" \
--metric MI[${fi},${mi},1,32] \
--convergence [100x100,1e-2,2] \
--shrink-factors 2x1 \
--smoothing-sigmas 1x0 \
--use-histogram-matching 1
<<<<<<<<<<<<<<<<<<<<<<<<
Fixed and moving images can be loaded from the ITK data files. What do you think about this example? Is it a bug in ITK or just the parameters are not tuned finely?
Thanks,
Ali
________________________________________
From: Johnson, Hans J
Sent: Tuesday, July 29, 2014 6:19 PM
To: Ghayoor, Ali
Subject: FW: [ITK-dev] [ITK] Optimizing composite transforms and center of transform
Ali,
You need to follow this discussion.
Hans
On 7/29/14, 6:16 PM, "Bradley Lowekamp" <blowekamp at mail.nih.gov<mailto:blowekamp at mail.nih.gov>> wrote:
Hello,
Ok, to further explore this problem, and hopeful help other people too: I
created an IPython Notebook viewable here:
http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/bl
ob/master/Transform%20Composition%20And%20Order.ipynb
It doesn't come through on the static page, but I added some nifty
interactive slider widgets to change the rotation parameter. Which is
really the point to enable understanding of the parameters for the
transform and how they should be best optimized. The notebook should work
with SimpleITK >= 0.8 and IPython Notebooks 2.1. I'll encourage every one
to try it out :) IPython Notebooks widget are very cool!
EXPLANATION:
The the order of the composite transform is that the new transforms are
applied first.
BUT they map from the virtual domain ( fixed ) to the moving image.
Therefore if your first transform moves the center of mass to the origin
then the second added transform's space will be centered on the mass,
which is good for optimizing rotation and affine parameters.
QUESTION:
Therefore I am inclined to conclude that its best practices to map the
fixed and moving image to the virtual domain such that the center of mass
( or some other "center feature" ) are at the origin. This would then the
scale, rotation and other affine parameters around the center, with out
having to used the "Center" parameter the transforms currently have.
Is this what others are doing in their registration? For general best
practices should this be recommend?
@Brian I know I have seen this multi-domain description many time, but I
think I may have just gotten it... Is the well describe in the literature
some place? I think this may be an important part add the software guide.
Thanks,
Brad
On Jul 29, 2014, at 5:38 PM, Matt McCormick <matt.mccormick at kitware.com<mailto:matt.mccormick at kitware.com>>
wrote:
Hi Brad,
Your assessment that the fixed center is important is correct.
In most cases, a transform with a Center is the first transform in a
Composite transform. There not any issues here. However, if the
transform with a Center is further along the chain, then a
CenteredTransformInitializer that was applied before the registration
is started will not generate the desired result. In this case, we
would have to respond to an Event in the registration process and
estimated the center on the imaged after the other transforms have
been applied. It is more work, but it is a more unusual case.
HTH,
Matt
On Mon, Jul 28, 2014 at 10:11 PM, Bradley Lowekamp
<blowekamp at mail.nih.gov<mailto:blowekamp at mail.nih.gov>> wrote:
Helloo Nick!
I am glad you got back to me.
I suspect that I have spent more time this past week looking at the
ITK affine transforms than anyone else[1] this week. With that here is
my current understanding...
1) The CenterTransformInitializers estimates an initial "Center" and
an initial "Translation" parameter from either the geometry or the
first moments. From my problematic experiences, getting the initial
transform which is a "good" guess is been critical for the optimizer to
head to the correct solution.
2) The Center parameter of a transform can either be "Fixed" or an
optimizable parameter. The transforms with the "Centered" monkier are
the the ones with the Center parameter optimizable. This issue seems to
be what you were referring to in to message.
In many ways the point I am trying to make is independent of #2. I
have observed that it's important to have the "Center" of an affine
transform ( or sub parameterization thereof ) at the center of the
object you are trying to register. That is the coordinate space of the
optimizable parameters' point 0 is near the center. This enables
rotation to easily rotate around the center, as well as scaling to
maintain the same relative center when "zooming". This issue is
independent of wether that center point can be optimized.
For example, consider changing the origin of an image for alignment by
say 500, and compensating with just an initial translation and not
setting the "Center" parameter. If we are trying to optimize rotation
and translation, then the optimization path would be very difficult to
traverse with these inter-dependent parameters to force it to rotate
around the center of the object. This scenario may be more common in
microscopy then medical imaging, due to microscopy frequently having
multiple subjects across large images. I could write up a IPython
Notebook to illustrate the case.
So that is my understanding of why using a fixed center is important.
I thought this may have been shared knowledge, but perhaps is not or is
incorrect...
Now I am trying to understand how this interacts with all the
coordinate frames involved with the ITKv4 registration framework, and
the composite transforms. Specifically the composite transform apply
the transforms in "reverse order"[2]. As I understand that that means
that the newest transform get applied first. So that transform
parameters are being optimized in the images space not in the virtual
domains we are working towards. Therefore the center of our transforms
is still important as we composite transforms?
I hope I explained that clearly, a white board may really be needed....
Thanks,
Brad
[1]
https://github.com/SimpleITK/SimpleITK/compare/master...9bc9bc8440e64d94
099d3519eb23bcc16c7cf015
[2] http://www.itk.org/Doxygen/html/classitk_1_1CompositeTransform.html
On Jul 28, 2014, at 8:57 PM, Nicholas Tustison <ntustison at gmail.com<mailto:ntustison at gmail.com>>
wrote:
Hi Brad,
I apologize for the delayed response. I¹m still catching up on my
workload after moving to CA. We might want to talk to Luis as it
was my understanding that the ³center² component of the linear
transforms might be a carry-over from all the ³Centered² transforms,
e.g.,
http://www.itk.org/Doxygen/html/classitk_1_1CenteredAffineTransform.htm
l
which, if I remember correctly, Luis said were somewhat sub optimally
conceived but this was such a long time ago that I might be completely
misremembering. However, fixing the center for all transforms within
a composite transform is certainly not necessary within the specified
framework. Rather, the matrix and offset are updated with the
³Center²
being updated implicitly.
Nick
On Jul 24, 2014, at 7:27 AM, Bradley Lowekamp
<blowekamp at mail.nih.gov<mailto:blowekamp at mail.nih.gov>> wrote:
Hello,
There are a lot of transforms involved in the new registration
framework.
I am trying to figure out what the implications for the fix
parameters of the transform "Center" ( ie center of rotation/scaling
) are when combined with composite transforms and the
fixed/moving/registration coordinate frames...
Based on how the transforms are composed it seems necessary to set
the fix "Center" for subsequent transformations. Additionally, I am
unsure how one would "improve" ( poorly defined? ) the center for a
subsequent transform ie Affine after a similarity...
Are there some documented guidance or figures to help with this
issue?
Do we have a comprehensive diagram of these transforms?
Thanks,
Brad
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php
Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers
_______________________________________________
Community mailing list
Community at itk.org<mailto:Community at itk.org>
http://public.kitware.com/mailman/listinfo/community
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Kitware offers ITK Training Courses, for more information visit:
http://kitware.com/products/protraining.php
Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-developers
________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140805/cb1de653/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unknown.png
Type: image/png
Size: 15787 bytes
Desc: unknown.png
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140805/cb1de653/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unknown.png
Type: image/png
Size: 15016 bytes
Desc: unknown.png
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140805/cb1de653/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MultiStageImageRegistration1.cxx
Type: application/octet-stream
Size: 35511 bytes
Desc: MultiStageImageRegistration1.cxx
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140805/cb1de653/attachment-0001.obj>
More information about the Insight-developers
mailing list