[ITK-dev] [ITK] Optimizing composite transforms and center of transform
Simon Alexander
skalexander at gmail.com
Tue Aug 5 09:34:40 EDT 2014
Hi all,
I have recently been playing around with this issue also. Has your hangout
happened already? Is there a summary of conclusions planned or existing?
Thanks!
On Tue, Aug 5, 2014 at 9:03 AM, Bradley Lowekamp <blowekamp at mail.nih.gov>
wrote:
> Ali,
>
> Glad to hear the registration iteration graphs where effective.
>
> Yes, you are right you can initialize the parameters of the second stage
> affine registration. Specifically the Center. However, it not readily
> apparent ( no example, no filter/initializer ) how that center fixed
> parameter should be initialized. And the "CenteredTransformInitializer" is
> not the correct way.
>
> For a Composite transformed defined as ϕ=T0(T1(x)) where ϕ:Ωfixed→Ωmoving then
> if C such that C∈Ωmoving is the correct center initialization for T1,
> then the correct center initialization for T0 when in the composite
> transform is C′=T−10(C).
>
> Cool pasting from the IPython Notebook with inline latex appears to have
> worked :)
>
> While it is easy to do this with the MomentsCaluclator and manually doing
> the transform inverse, I think to make it more readily apparent and easier
> to use it should be available as a filter/initializer.
>
> There are several things that are done manually in the examples that I
> would think should be updated to use the latest filters and initializers.
> For example The BSplines are manually initialized instead of using the
> BSplineTransform Initializer[1]. Additionally, conversion of BSpline to a
> deformation field[2], and the the upsampling of the BSpine[3], though that
> last one still looks like ITKv3 framework, so maybe it'll be updated to the
> adaptors with v4. However I am after simple functions for these common
> tasks for SimpleITK, so I have a bit different perspective then one who is
> OK with implementing a 50 line solution in C++ and being OK with the
> interface.
>
> We'll discuss these initialization issue further this afternoon.
>
> Brad
>
> [1]
> https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration4.cxx#L215-L235
> [2]
> https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration4.cxx#L401-L430
> [3]
> https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration6.cxx#L273-L330
>
> On Aug 4, 2014, at 11:56 PM, Ghayoor, Ali <ali-ghayoor at uiowa.edu> wrote:
>
> Hello Brad,
>
> It is an awesome way for demonstration, and thanks for verification of
> my example. It seems that this email is sent on Saturday, but unfortunately
> I got that this morning! And I should inspect the reason!
> Anyway, over the weekend I ran almost the same experiment in ITK and got
> close results to you.
>
> Also, today I tried to make the registration process work when Affine
> Transform is used in a multi stage structure, and here is my suggestion:
>
> I think we do not need to bother ourselves to think about a new
> optimizer or a new initializer for the composite transform since we can
> initialize the fixed parameters of each transform individually at the
> beginning of each stage.
>
> Remember how we run a BSpline registration stage after some linear
> registrations. At the beginning of the BSpline stage, we should instantiate
> a BSpline transform object and define its fixed parameters (grid size,
> etc). Then, we pass this initialized transform to the registration filter
> using “SetInitialTransform()”, yet we can initialize this stage by passing
> a composite transform, resultant from the previous stages, to the
> registration filter by “SetMovingInitialTransform()”.
>
> The same condition holds for all other transform types with fixed
> unoptimizable parameters like the center of rotation in our Affine
> transform case. Attachment file includes a multi stage registration example
> that I have created for the new software guide. In this example, we use a
> simple 2 stages registration process for a multi modal problem that moving
> image is misaligned from the fixed image by a translation shift and
> rotation. In registration process, a translation transform is followed by
> an affine transform. Also, an initial transform is used at the beginning.
>
> If I do not set the center of the affine transform, the registration
> fails as you have shown in your Ipython example. However, if at the
> beginning of the second stage, I compute the geometrical center (or center
> of the mass) of the fixed image, and pass that to the affine transform
> using “SetCenter()” or “SetFixedParameters()”, the registration succeeds.
> Please take a quick look in the attached file that also includes lots of
> explanations.
>
> Therefore, we can follow the same procedure for all other transform
> types that their fixed parameters are important in the optimization, but
> those fixed parameters cannot be set explicitly inside a composite
> transform. Please let me know with you think about this.
>
> Thank you,
> Ali
>
>
> From: Bradley Lowekamp <blowekamp at mail.nih.gov>
> Date: Saturday, August 2, 2014 at 6:48 PM
> To: Ali Ghayoor <ali-ghayoor at uiowa.edu>
> Cc: Hans Johnson <hans-johnson at uiowa.edu>, Insight Developers List <
> insight-developers at public.kitware.com>
> Subject: Re: [ITK-dev] [ITK] Optimizing composite transforms and center
> of transform
>
> Hello Ali,
>
> I created a notebook to try to demonstrate the problem you are
> describing:
>
> http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/master/Importance%20of%20Center%20with%20Affine%20Registration.ipynb
>
> If the center of an Affine transformation isn't correctly initialized
> then the optimization is not speedy or robust. I think it's best
> illustrated with these two graphs:
>
> <unknown.png>
> <unknown.png>
>
> Notice that with the Center being the origin, all 1000 iterations are
> exhausted, and the metric is slowing decreasing. And this is with the
> translation path being around the length to 10. If the spacing was bigger
> or more offset from the origin the situation would be worse.
>
> I don't think this is a bug with the registration framework. I think new
> initializers for composite transforms are needed, along with documentation
> how the 2 good ways to initialize the registration methods transforms.
>
>
> Brad
>
> On Aug 1, 2014, at 8:08 PM, Ghayoor, Ali <ali-ghayoor at uiowa.edu> wrote:
>
> Brad,
>
> I have exactly the same issue. I wrote a simple registration program that
> is run in two stages:
> -Translation
> -Affine
>
> My fixed and moving images are in 2D, and the moving image is created from
> the fixed image by
> rotation of 10 degree and translation of [13,17] in X and Y directions.
>
> The first stage does a translation registration and the result transform
> is added to a composite transform
> that is used as the initial moving transform of the second stage.
>
> After the registration the result transform of the second stage is also
> added to the composite transform
> to be used by the resampler.
>
> I have printed the composite transform to the screen:
>
> 2481: CompositeTransform (0x7fa359a18b00)
> 2481: RTTI typeinfo: itk::CompositeTransform<double, 2u>
> 2481: Reference Count: 2
> 2481: Modified Time: 20949
> 2481: Debug: Off
> 2481: Object Name:
> 2481: Observers:
> 2481: none
> 2481: Transforms in queue, from begin to end:
> 2481: >>>>>>>>>
> 2481: TranslationTransform (0x7fa359a199b0)
> 2481: RTTI typeinfo: itk::TranslationTransform<double, 2u>
> 2481: Reference Count: 7
> 2481: Modified Time: 1648
> 2481: Debug: Off
> 2481: Object Name:
> 2481: Observers:
> 2481: none
> 2481: Offset: [12.801, 15.8578]
> 2481: >>>>>>>>>
> 2481: AffineTransform (0x7fa359d095a0)
> 2481: RTTI typeinfo: itk::AffineTransform<double, 2u>
> 2481: Reference Count: 5
> 2481: Modified Time: 20946
> 2481: Debug: Off
> 2481: Object Name:
> 2481: Observers:
> 2481: none
> 2481: Matrix:
> 2481: 1.01593 -0.0100051
> 2481: -0.00835669 1.01113
> 2481: Offset: [-0.358829, -0.290093]
> 2481: Center: [0, 0]
> 2481: Translation: [-0.358829, -0.290093]
> 2481: Inverse:
> 2481: 0.984397 0.00974056
> 2481: 0.00813575 0.989073
> 2481: Singular: 0
> 2481: End of MultiTransform.
> 2481: <<<<<<<<<<
>
> As it can be seen, the translation transform provides a good
> initialization since its offset (Offset: [12.801, 15.8578]) is close to the
> shift parameters.
>
> However, the Affine transform fails to rotate the moving image since its
> Matrix is almost identity.
> Possible reason can be the improper Center of Affine transform that is
> fixed to [0,0], and not at geometrical center or center of mass.
>
> To make sure that I have not made any crazy mistake in my example, I
> simulated my example parameters in ANTs, and ANTs returned
> the same results and failed to register the images!!
>
> ANTs command line <<<<<<<<<<<
>
>
> PROGPATH=/scratch/ANTS/release-new/bin
>
> fi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainT1SliceBorder20.png
>
> mi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainProtonDensitySliceR10X13Y17.png
>
> time ${PROGPATH}/antsRegistration -d 2 \
> --float \
> --output [test1, warpedMoving.nii.gz] \
> --transform "Translation[16]" \
> --metric MI[${fi},${mi},1,32,None,1] \
> --convergence [100,1e-2,5] \
> --shrink-factors 3 \
> --smoothing-sigmas 2 \
> --use-histogram-matching 1 \
> \
> --transform "Affine[1]" \
> --metric MI[${fi},${mi},1,32] \
> --convergence [100x100,1e-2,2] \
> --shrink-factors 2x1 \
> --smoothing-sigmas 1x0 \
> --use-histogram-matching 1
>
> <<<<<<<<<<<<<<<<<<<<<<<<
>
> Fixed and moving images can be loaded from the ITK data files. What do you
> think about this example? Is it a bug in ITK or just the parameters are not
> tuned finely?
>
> Thanks,
> Ali
>
> ________________________________________
> From: Johnson, Hans J
> Sent: Tuesday, July 29, 2014 6:19 PM
> To: Ghayoor, Ali
> Subject: FW: [ITK-dev] [ITK] Optimizing composite transforms and center of
> transform
>
> Ali,
>
> You need to follow this discussion.
>
> Hans
>
> On 7/29/14, 6:16 PM, "Bradley Lowekamp" <blowekamp at mail.nih.gov> wrote:
>
> Hello,
>
> Ok, to further explore this problem, and hopeful help other people too: I
> created an IPython Notebook viewable here:
> http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/bl
> ob/master/Transform%20Composition%20And%20Order.ipynb
>
> It doesn't come through on the static page, but I added some nifty
> interactive slider widgets to change the rotation parameter. Which is
> really the point to enable understanding of the parameters for the
> transform and how they should be best optimized. The notebook should work
> with SimpleITK >= 0.8 and IPython Notebooks 2.1. I'll encourage every one
> to try it out :) IPython Notebooks widget are very cool!
>
> EXPLANATION:
>
> The the order of the composite transform is that the new transforms are
> applied first.
>
> BUT they map from the virtual domain ( fixed ) to the moving image.
> Therefore if your first transform moves the center of mass to the origin
> then the second added transform's space will be centered on the mass,
> which is good for optimizing rotation and affine parameters.
>
> QUESTION:
>
> Therefore I am inclined to conclude that its best practices to map the
> fixed and moving image to the virtual domain such that the center of mass
> ( or some other "center feature" ) are at the origin. This would then the
> scale, rotation and other affine parameters around the center, with out
> having to used the "Center" parameter the transforms currently have.
>
> Is this what others are doing in their registration? For general best
> practices should this be recommend?
>
> @Brian I know I have seen this multi-domain description many time, but I
> think I may have just gotten it... Is the well describe in the literature
> some place? I think this may be an important part add the software guide.
>
> Thanks,
> Brad
>
>
>
>
> On Jul 29, 2014, at 5:38 PM, Matt McCormick <matt.mccormick at kitware.com>
> wrote:
>
> Hi Brad,
>
> Your assessment that the fixed center is important is correct.
>
> In most cases, a transform with a Center is the first transform in a
> Composite transform. There not any issues here. However, if the
> transform with a Center is further along the chain, then a
> CenteredTransformInitializer that was applied before the registration
> is started will not generate the desired result. In this case, we
> would have to respond to an Event in the registration process and
> estimated the center on the imaged after the other transforms have
> been applied. It is more work, but it is a more unusual case.
>
> HTH,
> Matt
>
>
>
> On Mon, Jul 28, 2014 at 10:11 PM, Bradley Lowekamp
> <blowekamp at mail.nih.gov> wrote:
>
> Helloo Nick!
>
> I am glad you got back to me.
>
> I suspect that I have spent more time this past week looking at the
> ITK affine transforms than anyone else[1] this week. With that here is
> my current understanding...
>
> 1) The CenterTransformInitializers estimates an initial "Center" and
> an initial "Translation" parameter from either the geometry or the
> first moments. From my problematic experiences, getting the initial
> transform which is a "good" guess is been critical for the optimizer to
> head to the correct solution.
>
> 2) The Center parameter of a transform can either be "Fixed" or an
> optimizable parameter. The transforms with the "Centered" monkier are
> the the ones with the Center parameter optimizable. This issue seems to
> be what you were referring to in to message.
>
> In many ways the point I am trying to make is independent of #2. I
> have observed that it's important to have the "Center" of an affine
> transform ( or sub parameterization thereof ) at the center of the
> object you are trying to register. That is the coordinate space of the
> optimizable parameters' point 0 is near the center. This enables
> rotation to easily rotate around the center, as well as scaling to
> maintain the same relative center when "zooming". This issue is
> independent of wether that center point can be optimized.
>
> For example, consider changing the origin of an image for alignment by
> say 500, and compensating with just an initial translation and not
> setting the "Center" parameter. If we are trying to optimize rotation
> and translation, then the optimization path would be very difficult to
> traverse with these inter-dependent parameters to force it to rotate
> around the center of the object. This scenario may be more common in
> microscopy then medical imaging, due to microscopy frequently having
> multiple subjects across large images. I could write up a IPython
> Notebook to illustrate the case.
>
> So that is my understanding of why using a fixed center is important.
> I thought this may have been shared knowledge, but perhaps is not or is
> incorrect...
>
> Now I am trying to understand how this interacts with all the
> coordinate frames involved with the ITKv4 registration framework, and
> the composite transforms. Specifically the composite transform apply
> the transforms in "reverse order"[2]. As I understand that that means
> that the newest transform get applied first. So that transform
> parameters are being optimized in the images space not in the virtual
> domains we are working towards. Therefore the center of our transforms
> is still important as we composite transforms?
>
> I hope I explained that clearly, a white board may really be needed....
>
> Thanks,
> Brad
>
>
> [1]
> https://github.com/SimpleITK/SimpleITK/compare/master...9bc9bc8440e64d94
> 099d3519eb23bcc16c7cf015
> [2] http://www.itk.org/Doxygen/html/classitk_1_1CompositeTransform.html
>
> On Jul 28, 2014, at 8:57 PM, Nicholas Tustison <ntustison at gmail.com>
> wrote:
>
> Hi Brad,
>
> I apologize for the delayed response. I¹m still catching up on my
> workload after moving to CA. We might want to talk to Luis as it
> was my understanding that the ³center² component of the linear
> transforms might be a carry-over from all the ³Centered² transforms,
> e.g.,
>
>
> http://www.itk.org/Doxygen/html/classitk_1_1CenteredAffineTransform.htm
> l
>
> which, if I remember correctly, Luis said were somewhat sub optimally
> conceived but this was such a long time ago that I might be completely
> misremembering. However, fixing the center for all transforms within
> a composite transform is certainly not necessary within the specified
> framework. Rather, the matrix and offset are updated with the
> ³Center²
> being updated implicitly.
>
> Nick
>
>
>
> On Jul 24, 2014, at 7:27 AM, Bradley Lowekamp
> <blowekamp at mail.nih.gov> wrote:
>
> Hello,
>
> There are a lot of transforms involved in the new registration
> framework.
>
> I am trying to figure out what the implications for the fix
> parameters of the transform "Center" ( ie center of rotation/scaling
> ) are when combined with composite transforms and the
> fixed/moving/registration coordinate frames...
>
> Based on how the transforms are composed it seems necessary to set
> the fix "Center" for subsequent transformations. Additionally, I am
> unsure how one would "improve" ( poorly defined? ) the center for a
> subsequent transform ie Affine after a similarity...
>
> Are there some documented guidance or figures to help with this
> issue?
> Do we have a comprehensive diagram of these transforms?
>
> Thanks,
> Brad
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
> _______________________________________________
> Community mailing list
> Community at itk.org
> http://public.kitware.com/mailman/listinfo/community
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
>
>
> ________________________________
> Notice: This UI Health Care e-mail (including attachments) is covered by
> the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is
> confidential and may be legally privileged. If you are not the intended
> recipient, you are hereby notified that any retention, dissemination,
> distribution, or copying of this communication is strictly prohibited.
> Please reply to the sender that you have received the message in error,
> then delete it. Thank you.
> ________________________________
>
>
> <unknown.png><unknown.png><MultiStageImageRegistration1.cxx>
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/insight-developers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20140805/2e54fd99/attachment-0001.html>
More information about the Insight-developers
mailing list