[Insight-developers] benchmarking biomedical algorithms

[Holmes, David R. III, Ph.D. [RO RM1-24]]

Colleagues-

 

I apologize if this note is misplaced.  Please feel free to disregard
completely; however, I have been asked by a hardware manufacturer to
help determine what a "benchmark suite" of biomedical imaging algorithms
would look like.  I, personally, believe that this developer list is
viewed by the top developer, and I would appreciate your input.

 

My eventual goal for this is to spec out a list of algorithms to be used
in test suite.  The algorithms would have to include specific reference
which describe them and representative test data (image data and final
result).  This is somewhat similar to the dashboard testing, but the
data must be real, representative data.  It would be ideal if the
algorithms include code or pseudo-code.  I am hoping to draw the data
from the NLM-Mayo data collection as it is freely available.  For some
algorithms, I am hoping to draw directly from ITK; although in some
cases the generic programming/templated code may not be well-suited for
this type of benchmarking.

 

I would hope that this well-defined collection would be used, at a
minimum, to benchmark new CPU/GPU technologies; however, this would also
serve as a nice benchmark to test out different implementations of
algorithms.  I know that this is something that the ITK community has
repeatedly talked about, but I don't yet know of a test suite.

 

I plan to span a wide variety of algorithms, and I would initially like
to avoid very complex algorithms (or stochastic algorithms) as they may
encumber the purpose of the suite (e.g. benchmarking). Rather, I am
interested in basic building blocks which would be incorporated into
more complex and valuable algorithms.   I'm after algorithms which
require different numerical precision, different regions of support,
different dimensionality, etc.  For example, I would by default include:

 

-Thresholding

-Erode or Dilate Morphologic operation

-Volume transformation

-simple volumetric projection - MIP or summed-voxel projection

-linear or non-linear filtering - average, rank, gradient filter.

-histogram calculation

 

Anyway, I would appreciate any input that you might have.  I guess I
should really first ask if someone has already developed such a suite.
Again, I apologize if this is not the right forum.

 

Thanks

 

david

 

============================================

David R Holmes III

Department of Physiology and Biomedical Engineering 

Mayo Clinic College of Medicine 

 

Ph:  507-266-4250       Fax:  507-284-1632 

holmes.david3 at mayo.edu

============================================ 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.itk.org/mailman/private/insight-developers/attachments/20081106/d94bd539/attachment.htm>