[Insight-users] Not so Silly questions about registration

[Lydia Ng]

Hi David,

At the risk of adding fuel to the fire ... here is my 2 cents worth of
personal comments (others within the ITK contractors may differ).

It seems to me there are two separate issues in your email:

[1] Why doesn't the registration framework work with any input image
type?

[2] Why doesn't one set of parameters (e.g. for the optimizer, metric)
work for every registration problem?

-----------------------------
I agree with your point of view of [1] and we will endeavor to try to
resolve that.

While [2] I believe is an application problem.

The scope of ITK was to build a toolkit with generic component that can
be reuse. I believe that we have mostly achieved this with the
registration framework. What you are talking about is the "smarts" in
connecting the component together so that it would work for any
registration problem you give it? I believe that is more of "R" effort,
while ITK is a "D" effort intended to efficiently support the "R"
efforts.

As you probably know, that there are little tricks one can do, normalize
the images, center the images etc. In term of encapsulation these tricks
should probably lie outside of the registration components. This would
allow researchers to try out their tricks.

The examples in the software guide were more aim to be a tutorial on how
the registration framework components can be put together - and
shouldn't be view as "registration black-boxes". Mind you, if people
want to contribute black-box registration applications to
InsightApplications they are very welcome.

------------------------------

Parameter selection seems like the bugbear of complex image processing
algorithms in general and I too feel your frustration.

 - Lydia


> -----Original Message-----
> From: David Holmes [mailto:holmesd3@yahoo.com]
> Sent: Tuesday, February 25, 2003 8:52 AM
> To: Luis Ibanez
> Cc: insight
> Subject: Re: [Insight-users] Not so Silly questions about registration
>=20
> Luis-
>=20
> I appreciate your feedback.  In response to your first
> note, I happen to agree more with Dan on this issue.
> Just because your input datatype is unsigned char,
> there is no reason that everything processed within
> that object needs to be unsigned char.  Internal
> processing which requires floats should be carried out
> in floats.  For example, there is no reason that
> within the framework, the gradient info couldn't be
> calculated and stored as a float regardless of the
> input datatype.  I think that this is a case where I
> was possibly too pixeltype aware -  (i.e. the pixel
> data is unsigned char, not float).  Although not a
> naive user, I take the mindset of a good
> object-oriented program.  I pass my stuff to a object
> and wait for a response.  It is not necessarily my job
> to dig into someone elses class and determined the
> process step by step.  In fact, I should be able to
> answer all of my question about a class from the
> header and public/protected methods defined -- as well
> as the documentation.  In particular, I expect that
> the object (i.e. the registration object) should take
> my data as is and deal with it accordingly.  If it
> cannot handle my data, then that should be documented.
>  I was unable, at first glance, to see any
> documentation that stated a simple transform
> registration with the regularstepgradientoptimizer
> will only work with float data.  Instead, I would have
> expected the registration object to resolve that
> problem for me.  If this is not the case, then I would
> have to dig over every piece of code to determined
> every programming detail of the algorithm which means
> I would have to dig into the methods of the classes
> which don't fit well with my understanding of OO
> concepts such as encapsulation.  I realize that this
> is somewhat of an extreme viewpoint, and I will admit,
> I don't complete adhere to it, but I think that it
> makes the point.  In this particular case, I think
> that the issue is that I was unable to find the
> supporting documentation without diggging into a lot
> of code.
>=20
> With regards to your second note, I think it is
> unreasonable to use your simple example.  Rarely is
> anyone going to divide two int and store it in an int.
>  That just doesn't make sense.  The correct response
> to dividing two int is to type one of them and store
> as a float.  Once again, this is something that could
> be done within a class rather than requiring
> everything be the same type from the beginning.  In
> the same manner, to make an image of type std:string
> doesn't make sense.  It is a matter of common sense.
> There is nothing in "common sense" thinking that says
> that I shouldn't be able to pass two unsigned chars
> into a registration framework and work towards an
> achieveable goal (even if it is not identical), but as
> the results show that a reasonable solution cannot be
> found.  In contrast, there is no common sense to
> dividing two int and storing in an int if precision is
> an issue or creating an image of type std::string.
>=20
> In general, I think that ITK is a very nice toolkit.
> One of those features is the generic programming with
> templates, however, this does not imply that it is
> robust - as my example shows.  When reviewing the
> mailing list, there are several cases in which the
> answer to a question has been along the lines of "this
> example was tailored to the particular
> problem...because your application may be different,
> you must adjust the parameters to fit your specific
> problem."  In particular with an image guided surgery
> system, the software must be robust, because the
> programmer and user don't have all of the necessary
> time to create data specific solutions for every case
> - particularly in real-time as things are changing.  I
> admit some small amount of tweaking is necessary for
> determining the optimal solution, but a programmer or
> user shouldn't have two write a completely new program
> for each specific problem.
>=20
> To summerize:
>=20
> (1) I like ITK
> (2) I am sorry to have complained so much
> (3) I do appreciate the help that you have given me
> (4) I recognize that we all have different
> perspectives on how we solve problems.  I think the
> key is to try and find the best solution for every one
> (5) I think that, in this case, there may be a need
> for further documentation
>=20
>=20
> Thanks
>=20
> david
>=20
> --- Luis Ibanez <luis.ibanez@kitware.com> wrote:
> >
> > Hi Daniel,
> >
> > I'm affraid to have to disagree.
> >
> > It doesn't make perfect sense to replace floats with
> > chars
> > and expect to get the same results, not even with
> > simple
> > operations.
> >
> > Let's consider:
> >
> >      T a;
> >      T b;
> >      T c =3D a * b;
> >      T d =3D a / b;
> >      T e =3D a - b;
> >      T f =3D a + b;
> >
> > Can we expect this to work the same for T=3Dchar and
> > T=3Dfloat ?
> >
> > T=3Dfloat and T=3Ddouble will already give different
> > results...
> >
> > and if you use VC++, T=3Dfloat in debug mode and
> > T=3Dfloat in
> > release mode will also give different results.
> >
> > Even if we use the NumericTraits<> and cast to the
> > computational
> > types, there are always lateral effects the we have
> > to care about.
> >
> > Do we want to discover these secondary effects at
> > run time in
> > an image guided surgery system ?
> >
> > or do we want the developers to deal with the
> > problem at compile
> > time when they can afford the time to analyze the
> > problem and fix
> > it while having enough coffee available.
> >
> > If we put ourselves in the shoes of a naive user of
> > Insight,
> > the first thing we should do is to take the naivety
> > out of him.
> > Naivety is not a desirable quality for a developer
> > of medical
> > image applications.
> >
> > Insight is not only a software toolkit, it is also
> > an educational
> > resource. I think we should be commited to insist in
> > the special
> > requirements of medical imaging as opposed to plain
> > computer
> > graphics concepts. Insight is not Photoshop. The
> > images that we
> > are intended to process contain information about
> > somebody lying
> > on a bed in a hospital. Taking care of the
> > implications of data
> > types doesn't seem to be such a burden when we
> > consider it in
> > this perspective.
> >
> > --
> >
> > Unfortunately we cannot restrict the registration
> > classes to work
> > only on float images. The reason is that some
> > registration problems
> > can actually be solved in char pixel types.  It
> > depends on the metric
> > we use, and users have the freedom to define their
> > own metrics.
> >
> > As in real life, Freedom comes with Responsibility
> > attached.
> >
> > Insight is a toolkit, there is a limit to the things
> > we can do
> > for preventing somebody from using a hammer to take
> > out a screw.
> > We could install an anti-screw security system in
> > the hammer but
> > this will probably make the hammer useless for
> > hitting nails.
> > This is, in my opinion, more of a documentation and
> > education
> > issue than a programming one.
> >
> > As developers we cannot be charged with getting the
> > same results
> > across a range of datatypes. That is simply not
> > possible. Can
> > we add two constant images with values 250 in
> > unsigned char ?
> > There are, after all, good reasons for different
> > types to exist.
> >
> > The fact that we can instantiate it, doesn't mean it
> > will work.
> >
> > We can instantiate an image of PixelType =3D
> > std::string. No problem
> > at all, but we cannot expect a median filter to work
> > on this image,
> > nor to be able to save it in a PNG file.
> >
> > ----
> >
> > The GradientMagnitudeRecursiveGaussianImageFilter
> > requires so much
> > memory because it provides a trade-off between
> > memory and speed.
> > If memory becomes a problem, then you have the
> > option of paying in
> > speed and use a convolution based filter like the
> > GradientMagnitudeImageFilter. It will take 10 times
> > less memory
> > but it will probably run 10 times slower.
> >
> > ----
> >
> >
> > It is always healthy to revisit the frameworks and
> > verify if they
> > can be improved. That's after all the main advantage
> > of an Open
> > Source project. I will be glad to analyze the
> > options that could
> > lead to solve registration problems on a whole range
> > of data
> > types and discuss their possible implementations.
> >
> > Proabably the best way to start is to create the
> > trivial test you
> > suggested in which we could evaluate which types
> > behave correctly
> > for different registration setups.
> >
> >
> > Would you like to give it a try to implement this
> > test ?
> > I'll be happy to help.
> >
> >
> > Best regards
> >
> >
> >
> >     Luis
> >
> >
> >
> > ---------------------------------------------
> >
> > Blezek, Daniel J (Research) wrote:
> > > Luis, Dave,
> > >
> > >   I'm not sure this is the best way of addressing
> > Dave's concerns.  Let's put ourselves in the shoes
> > > of a naive user of Insight.  Dave did exactly what
> > I would have done.  He said, "Hmm, why waste 4x of
> > > my memory by using floats, I'll just run on
> > unsigned chars, and get the same result.  This makes
> > > perfect sense (to me at least).
> > >
> > >   IMHO, it's OK to be PixelType-Aware, but Dave
> > would have spent an amazing amount of time trying to
> > > figure out why he got different results changing
> > from float to unsigned char for registration.  While
> > > I agree that floats _can_ be faster, you can also
> > be killed if you start swapping.  If we really want
> > > to catch problems at compile time, shouldn't the
> > registration classes prevent you from instantiating
> > > a non-floating point class?  Again, as a naive
> > user, I would think one of the best benefits of ITK
> > is
> > > being able to create an unsigned char version of
> > MI registration if I want, but apparently, that is
> > > not the case.  Shouldn't the ITK developer be
> > charged with getting the same results across a range
> > of
> > > datatypes?  How hard would it be to make the
> > GradientRecursiveGaussianImageFilter have an output
> > of
> > > float, and do all internal calculations with
> > floats/doubles?
> > >
> > >   I was a bit put off that the
> > GradientRecursiveGaussianImageFilter requires 10x
> > the image size to do
> > > it's work.  That seems really huge.  For example,
> > if I have a 512x512x512 CT (268M, fixed) to
> > > register to a 256x256x128 MR image (16M, moving).
> > I'd end up having to convert to float ((268+16) *
> >
> =3D=3D=3D message truncated =3D=3D=3D
>=20
>=20
> __________________________________________________
> Do you Yahoo!?
> Yahoo! Tax Center - forms, calculators, tips, more
> http://taxes.yahoo.com/
> _______________________________________________
> Insight-users mailing list
> Insight-users@public.kitware.com
> http://public.kitware.com/mailman/listinfo/insight-users