[Insight-users] Optimizer's Exception in
MattesMutualInformationImplementation
Luis Ibanez
luis.ibanez at kitware.com
Wed Aug 4 13:20:34 EDT 2004
Hi Thomas,
It seems that you forgot to post the trace log
of one of the registrations that failed. If you
look carefully at those plots, they reveal what
sort of parameter tunning may help to drive the
optimization process to convergence.
Please understand that without having the chance
to look at your code, nor the metric plot, nor the
images, there is very few we could do to help you.
Except probably to give you generic philosophical
advice and some sort of fraternal moral support. :-)
For the philosophical advice:
Is there any particular reason why you
decided to use Mutual Information ?
Given the nature of Ultrasound Data,
it doesn't seem that Mutual Information
will be particularly better suited for
these images than any other metric.
I would suggest you to try the following
things:
1) Identify the *relevant* structures
that you want to register form your
images
2) Identify what is unique on those
structures.
3) Attempt to run pre-processing filters
on your images in order to enhance
the visibility of the features that
you care about registering
4) Experiment with other metrics such
and MeansSquares and Normalized
Correlation.
5) The typical cause for this exception
to be thrown is that the transform
goes so off that the overlap between
the images becomes minimal and the
number of samples that can be taken
for computing the metric are not
enough for providing a reasonable
estimation of Mutual Information.
6) The step length of the optimizer is
also a usual suspect when dealing
with this exception. Step lenght
or learning factors that are too
large result in unstable optimization
that can easily drive the transform
parameters too far from the capture
basin of the optimal value.
If you are particularly attached to
Mutual Information. You could also
include in your algorithm a recovery.
E.g. after catching the exception
restarting the transform in another
location.
Regards,
Luis
---------------------------
Thomas - Kuiran Chen wrote:
> Hi Luis,
>
> Thanks for the detailed advice. :-)
>
> I did have already used the command/observer to print output from the optimizer, and the results revealed that there are a few unexpected problems:
> 1. The exception is random: In our experiment, we've caught the image pairs that throwed the exception, but when we reran the exact same program but just for these assumed-problematic image pair, there were no such exception at all.
> 2. The error is not repeatable: the image pairs throwed the exception changed everytime we ran the program.
> 3. If you look at those image pairs that did throw the exception at one time and those didn't, the difference were very minimum: there is no much difference in image distribution, intensity distribution, or noise level, and were captured using the same US machine with the same setting. The reason being that is simple: we captured these images using freehand US probe in one experimental setup and within only period of 3 minutes. Further more, to test the registration algorithm, we moved the probe very slowly and in a very small range (with 3 centimeters) and no rotation/tilting of the probe, just simply translation.
>
> Thanks,
> Thomas
>
> ---------------------- Original Message ------------------------
> From: Luis Ibanez <luis.ibanez at kitware.com>
> To: Thomas - Kuiran Chen <chent at cs.queensu.ca>
> Sent: 2004-08-04 08:10:31
> Subject: Re: [Insight-users] Optimizer's Exception in Mattes MutualInformationImplementation
>
>
> Hi Thomas,
>
> Image Registration (and image segmentation) are
> not processes that are likely to work blindly.
>
> It is actually surprising that your generic setup
> worked for such a large fraction of your images
> without requiring further fine tunning.
>
>
> Please do the following:
>
> 1) Take one of the image pairs that fail the
> registration by throwing an exception
>
> 2) Connect a Command/Observer to the optimizer
> as it is illustrated in almost all the image
> registration examples in the ITK SoftwareGuide
>
> http://www.itk.org/ItkSoftwareGuide.pdf
>
>
> 3) From this command/observer print out the
> value of the metric and the transform
> parameters at every iteration of the optimizer
>
> 4) Post those results to the list
>
> 5) If you can post the two images also, that will
> be great. Our guess is that those images present
> a larger misrregistration than the image pairs
> that were successfully registered, and therefore
> initializing the transform as an identity transform
> is not appropriate since you start too far from
> the capture basin of the global optimum.
>
>
> Knowing about the large variety of image quality
> in ultrasound... it is not surprising either that
> some of your registration fails. It will be interesting
> to evaluate the degree of speckle content in your
> images, and eventually apply denoising filters on them.
>
>
> Are your images from 2D ultrasound ?
> or from 3D ultrasound ?
>
> If they are 2D, you should also check how the ultrasound
> acquisition plane changed from one image to the other.
>
> If you rotated the plane "out of the plane",... any
> registration will be a pure coincidence, and actually
> something that you may not want to use for Medical
> Applications since you will be matching anatomical
> structures that are not equivalent. E.g. a slice
> of liver is not "registrable" against a slice of
> liver cut at 20 degrees from the first one. Even
> if your program were successful in such registration,
> the usability of such result must be considered
> carefully.
>
> You will find interesting to look at the recent report
> on Medical Errors
>
> http://tmlr.net/jump/?c=9593&a=296&m=2596&p=914464&t=164
> http://app2.topiksolutions.com/ct_track.ts?c=17778&u=914464&i=2596&ct_attrib=http%3A%2F%2Fwww.healthgrades.com%2Fmedia%2Fenglish%2Fpdf%2FHG_Patient_Safety_Study_Final.pdf
>
> Quote:
>
> "The number of deaths from medical errors in the U.S.
> could run as high as 195,000 a year, according to a
> new analysis done by the healthcare rating company
> Health Grades Inc., a number nearly double the worst-
> case estimate of the Institute of Medicine's landmark
> 1999 "To Err is Human" report."
>
>
>
> I assume that you are using ultrasound for diagnosis,
> and/or providing guidance in minimally invasive surgery.
> In both cases you should take a very critical look to
> the entire registration process in order to make sure
> that the mathematical results have some medical sense.
>
>
>
> Regards,
>
>
> Luis
>
>
>
> -----------------------------
> Thomas - Kuiran Chen wrote:
>
>
>>Hi Luis,
>>
>>Thanks for the message.
>>
>>I did catch and print out the error message coming out of the optimizer:
>>
>>ITK ExceptionObject caught !
>>itk::ExceptionObject (0105F1F0)
>>Location: "Unknown"
>>File: c:\thomas_k_chen\data_research\projects\thirdparty\itk-1.6.0\code\algorithms\itkMattesMutualInformationImageToImageMetric.txx
>>Line: 546
>>Description: itk::ERROR: MattesMutualInformationImageToImageMetric(0180FEA0): Too many samples map outside moving image buffer: 1535 / 6581
>>
>>What confused me is, if I test individual two images to perform the registration using Mattes mutual information, there is no exception at all. But when I used a for-loop to compare images pair by pair, I got exceptions thrown randomly (sometimes happened to this pair of images, sometimes the other):
>>
>>e.g.:
>>
>>for(counter)
>>{
>> // 2000 pairs of images to compare
>> mattes mutual information registration wrapper.
>>}
>>
>>Now I bypassed this problem by using:
>>
>>for(counter)
>>{
>> // 2000 pairs of images to compare
>> try
>> {
>> mattes mutual information registration wrapper.
>> }
>> catch(e)
>> {
>> reduce the counter;
>> continue;
>> }
>>}
>>
>>This way I was able to complete the loop, but still what caused the exception remains a mystery to me.
>>
>>The initial transform shouldn't be the culprit because it starts with identiy transform (zero translations).
>>
>>What I suspect is that the sampling points (for both images) are generated on a random base, and in some case, the picked points are not overlapping each other enough.
>>
>>Thanks a lot,
>>Thomas
>>
>>
>>
>>---------------------- Original Message ------------------------
>>From: Luis Ibanez <luis.ibanez at kitware.com>
>>To: Thomas - Kuiran Chen <chent at cs.queensu.ca>
>>Sent: 2004-08-03 00:32:27
>>Subject: Re: [Insight-users] Optimizer's Exception in Mattes Mutual InformationImplementation
>>
>>Hi Thomas,
>>
>>
>>It is unlikely for optimizers to throw exceptions. If you are getting
>>some, chances are that they are comming from the ImageMetrics.
>>
>>The typical case is when the transform is so far off the mark that
>>the two image are not overlapped at all, or overlap in a very small
>>region. In those circumstances the metric is not able to compute
>>meaninful values and derivatives.
>>
>>
>>NOTE you will understand better the process if you print out the
>>description returned by the exception when you are inside the
>>catch block. It will look like
>>
>> > try
>> > {
>> > // do the optimization
>> > m_Optimizer->StartOptimization();
>> > }
>> > catch( ExceptionObject& err )
>> > {
>> > // An error has occurred in the optimization.
>> > // Update the parameters
>> > m_LastTransformParameters = m_Optimizer->GetCurrentPosition();
>>
>> // PLEASE ADD THIS LINE
>> std::cerr << err << std::endl;
>>
>> >
>> > // Pass exception to caller
>> > throw err;
>> > }
>> >
>>
>>
>>The problem itself is not in the size of the histograms used
>>for estimating mutual information, but in the initialization
>>of the transform and the step size used by the optimizers.
>>
>>If your transform is going off the overlapping regions,
>>it is likely that your optimization steps are too large,
>>and/or your initial transform is quite a bad initial position.
>>
>>
>>
>>Regards,
>>
>>
>>
>> Luis
>>
>>
>>----------------------------
>>Thomas - Kuiran Chen wrote:
>>
>>
>>
>>>Hi Luis,
>>>
>>>I am running into another interesting problem while using the Mattes' mutual information implementation with ultrasound images in ITK. :-)
>>>
>>>My question is: in which circumtances, will the optimizer throw an exception?
>>>
>>>In particular, for the Mattes implementation, the only change I made is I switched from one set of input images to another set of input images, while both sets are ultrasound images. For one set of the inputs, the optimizer throwed the exception in the following position (with err = 0):
>>>
>>>itkImageRegistrationMethod.txx:
>>>
>>>try
>>> {
>>> // do the optimization
>>> m_Optimizer->StartOptimization();
>>> }
>>> catch( ExceptionObject& err )
>>> {
>>> // An error has occurred in the optimization.
>>> // Update the parameters
>>> m_LastTransformParameters = m_Optimizer->GetCurrentPosition();
>>>
>>> // Pass exception to caller
>>> throw err;
>>> }
>>>
>>>The details symptoms are:
>>>
>>>1. When I used 1st set of ultrasound images (2000 of them, each of 381x343 pixels), histogram bin size = 50, sampling rate = 1%, the Mattes mutual information implementation ran properly, without any programatic or runtime errors;
>>>
>>>2. Then I used 2nd set of ultrasound images (2000 of them, each of 191x341 pixels), histogram bin size = 50, sampling rate = 1% or 10%
>>>- if I just use 200 images out of the 2000, the Mattes implementation worked with no problems (just as in 1);
>>>- if I choose to use more than 300 images out of the 2000, the above exception was thrown, with err = 0; I even increased the sampling rate to 30%, but the problem persisted.
>>>
>>>The two sets ultrasound images were captured using the same machine and same settings, but on different subjects: one with the soft tissues and one without. The problem occured when I used the images of soft tissues. But the only difference that I can tell between these two image sets are:
>>>- image size, and
>>>- different histogram distribution (images of soft tissue have more histogram variance).
>>>
>>>What confused me is all of my 2000 ultrasound are basically similar in either intensity distribution or noise levels, but in some case (like few images through the loop) the program worked, but it didn't whe I increased the number of input images through the loop.
>>>
>>>Kindly please enlighten me, :-).
>>>
>>>Thanks very much!
>>>
>>>Thomas Kuiran Chen
>>>
>>>
>>>_______________________________________________
>>>Insight-users mailing list
>>>Insight-users at itk.org
>>>http://www.itk.org/mailman/listinfo/insight-users
>>>
>>
>>
>>
>>
>>
>>
>>--------------------------------------------------------------------
>>
>>
>>
>>
>
>
>
>
>
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org
> http://www.itk.org/mailman/listinfo/insight-users
>
> --------------------------------------------------------------------
>
>
>
> _______________________________________________
> Insight-users mailing list
> Insight-users at itk.org
> http://www.itk.org/mailman/listinfo/insight-users
>
More information about the Insight-users
mailing list