[Insight-developers] Image file formats for testing
Daniel J. Blezek, Ph.D.
blezek@crd.ge.com
Mon, 26 Mar 2001 10:38:07 -0500 (EST)
Parag and Stephen,
Before my life at GE, I worked with Dr. Rick Robb at Mayo on Analyze.
Again, the Analyze file format is simply a "roll-your-own" format, that
served the need of the group at the time. The current incarnation of
Analyze, AVW has a better model of a file format. The file is simply text
indicating spacing, scanner, datatype, etc... with filenames to read as
each slice. Thus, the individual slices are 2D but wrapped into 3D.
I'd like to make it clear that I'm not proposing a fileformat for all of
Insight to use forever, just a format to settle on for the purpose of
testing our code. I still feel that PNG makes sense. The specification
allows you to set the physical scale units(In Meters), and height and
width, and we can encode depth into the text comments that PNG allows.
I'm really more interested in PNG as an _output_ format, not necessarly as
an input format, e.g. a test reads whatever format it likes (assuming it's
a cross-platform library), does it's processing, and writes out a PNG
file. A later process picks up the PNG files, compares them to the "gold
standard" for that algorithm, producing difference images if the test
generated a different result. This would allow a developer to produce
input data in whatever (supported) format is most convient, and allow
everyone to view the testing output, and for the dashboard to easily show
the images.
On a related note, I've been considering a framework to allow Insight to
be more easily streamlined with other packages, sort of an import/export
facility, with a simplified pipeline interface. That would make using
Insight with VTK or other packages straightforward. In this design, it
makes sense to move the ImageIO class under the import/export classes, for
after all, file i/o is interaction with an external system. This should
be relatively straightforward, and not very invasive, but will involve
re-arranging the code a bit. I'm hoping to get some time to work on this
later this week, and I'll run my design by you both, before making changes
to CVS.
Thanks,
-dan
> The only difficulty is that PNG doesn't provide element spacing
> information. How about a medical format - e.g., analyze?
> Stephen
On Fri, 23 Mar 2001, Parag Chandra wrote:
> Daniel, I think this Toolkit::GetFileName() is exactly what we need, because
> there has to be some way to get at the contents of variables like
> ${CMAKE_BINARY_DIR}. Otherwise the test process will have to cd to the
> directory of the gold-standard images and then run each of the
> platform-specific binaries via its fully-qualified pathname.
>
> With regard to the file formats issue: since ImageIO is abstract, we have to
> use a concrete reader for a specific file format in order to test. I chose
> MetaImage because at one point I thought it was going to be one of the
> "officially" supported formats. If this is no longer true, then we need to
> decide on a format for testing purposes, and I think we need something more
> comprehensive than PNG, or perhaps any other single file format for that
> matter. 3D images are used frequently enough that I think 3D image I/O
> should be part of the test process, but I realize 3D images would be
> difficult to show in a browser for comparison. However, instead of a visual
> comparison, couldn't we just use some sort of binary diff command?
>
> -Parag
>
> ----- Original Message -----
> From: "Daniel J. Blezek, Ph.D." <blezek@crd.ge.com>
> To: "Stephen R. Aylward" <aylward@unc.edu>
> Cc: "Insight-Developers (E-mail)" <insight-developers@public.kitware.com>
> Sent: Friday, March 23, 2001 8:24 AM
> Subject: Re: [Insight-developers] [Fwd: Changes to MetaImage...]
>
>
> > Stephen and Parag,
> >
> > This exact issue drove the discussion about file reading and writing at
> > last week's telephone conference. I have several ideas in mind as to
> > finding images and writing images for regression testing, but have not
> > made them happen just yet. In the mean time, I would welcome any
> > suggestions you may have as to how to address this issue. In our
> > experience with VTK, which may be different for ITK, some platforms
> > require a platform specific "gold image", and require special processing.
> >
> >
> > Here are some of my ideas, strictly related to regression testing:
> >
> > - Have a Toolkit similar to Java, so to get the name of a file you would
> > do something like this:
> >
> > std::string s = Toolkit::GetFileName ( "VHF/Head.png" );
> >
> > The Toolkit would properly expand the file name to be the path to the
> > image you want, allowing images to be stored on fixed media(CDROM). On
> > the outbound side, you would request an output image in a similar manner:
> >
> > std::string s = Toolkit::GetImageTestResultFileName (
> > "itkFileIOMetaImageTest" );
> >
> > The Toolkit would expand this name to a directory where test images
> > sit, and a latter process would pick up the images and compare them to the
> > "gold standard". If the test were run in an example mode, it would simply
> > write the output in the current directory, for the user to take a look at
> > later.
> >
> >
> > - This raises the issue of file formats yet again. Everyone seems to cook
> > up their own file format to meet the needs of whatever problem they have
> > at hand! (We are no exception...) In my view, the current problem at hand
> > is regression testing. I've given this issue some thought. We need:
> >
> > - a file format that is easily displayed locally and through a browser
> > - a format that is easily created by all Insight developers
> > - a format that has 2D support at minimum
> > - a format that supports as many common datatypes as possible
> >
> > The first point drives the selection down considerably. The only real
> > canidates are Gif, Jpeg, and PNG. Gif and Jpeg are generally lossy
> > compression, and don't fit the bill. PNG is a relatively new format, with
> > good support from most OS/viewers/browsers. PNG is also licensed for use
> > in a toolkit such as ITK. The real downside is that PNG only supports 8
> > and 16 bit datatypes. I feel this is acceptable because medical image
> > data generally comes in these two types, so that should not be a limiting
> > factor. PNG support is built and working on my local checkout.
> >
> > Since I'm doing most of the testing infrastructure, I will offer to serve
> > as the focal point for this issue, and welcome any suggestions.
> >
> > Thanks,
> > -dan
> >
> >
> > On Thu, 22 Mar 2001, Stephen R. Aylward wrote:
> >
> > >
> > > How should we reference data files in the example directories if the
> > > build is placing the executables in different directories? See below
> > > for more details....
> > >
> > > Thanks,
> > > Stephen
> > >
> > > Parag Chandra wrote:
> > > >
> > > > Ok, that solves half the problem. I'll put the test and the
> accompanying
> > > > files into examples. But how would I reference that (the examples)
> directory from within
> > > > the test program? Each platform will place the executable for this
> test into
> > > > something like ${CMAKE_BINARY_DIR}/Examples, but ${CMAKE_BINARY_DIR}
> isn't
> > > > even an environment variable for me to reference, it's a variable
> that's
> > > > internal to CMake. If I reference the file without the pathname, like
> > > > "TestInput.mhd", then the file has to be in the same directory as the
> > > > executable, and that means I would have to place a copy of the test
> file
> > > > into Insight-Cygwin, Insight-VC++, Insight-Linux, etc. Not only is
> that
> > > > redundant, but there's no way I could foresee the additional platforms
> we
> > > > might target, and so the test running on those platforms will generate
> an
> > > > exception when it can't find the file. The only thing I can see is to
> place
> > > > the test file into one directory, and then run the test for each
> platform
> > > > from that directory by using the full pathname of the executable, e.g.
> > > > /Insight-Cygwin/Examples/itkFileIOMetaImageTest.exe. Does anyone have
> any
> > > > other ideas?
> > > >
> > > > Thanks,
> > > > -Parag
> > > >
> > > > ---
> > > > You are currently subscribed to caddlab as: aylward@unc.edu
> > > > To unsubscribe send a blank email to
> leave-caddlab-706N@listserv.unc.edu
> > >
> > > --
> > > ===============================================
> > > Stephen R. Aylward
> > > Assistant Professor of Radiology
> > > Adjunct Assistant Professor of Computer Science
> > > http://www.cs.unc.edu/~aylward
> > > aylward@unc.edu
> > > (919) 966-9695
> > >
> > > _______________________________________________
> > > Insight-developers mailing list
> > > Insight-developers@public.kitware.com
> > > http://public.kitware.com/mailman/listinfo/insight-developers
> > >
> >
> > --
> > Daniel Blezek, Ph.D.
> > blezek@crd.ge.com
> > Visualization and Computer Vision Program
> > Electronic Systems Lab
> > GE Corporate Research & Development
> >
> >
> > _______________________________________________
> > Insight-developers mailing list
> > Insight-developers@public.kitware.com
> > http://public.kitware.com/mailman/listinfo/insight-developers
> >
>
--
Daniel Blezek, Ph.D.
blezek@crd.ge.com
Visualization and Computer Vision Program
Electronic Systems Lab
GE Corporate Research & Development