[Insight-users] Re: SV: A question about point cloud

[Gavin Baker]

Hello Peter,

> Peter Stevrin wrote:
> 
> > My name is Peter Stevrin and I have a project regarding 3D modelling.
> > It is to make a 3D model of the human ear canal from digital
> > images. The device used consists of a camera, an adapter, and
> > otoscope. With this instrument are images captured of the ear
> > canal. This work is done in a hearing clinic. The images are then
> > stored in a computer. Before the images are sent over Internet to a
> > manufacturer of hearing aids the images are filtered and
> > compressed. When images arrive to the manufacturer they are
> > decompressed and filtered. The capturing, filtering compression and
> > decompression are done with software developed for this application.
> >
> > The problem that we now must solve is to track the digital images in
> > order to get a point cloud based on points and coordinates. Do you have
> > a solution to this problem?

Rasmus Paulsen has been working on the very problem of modelling the human
ear canal.  His homepage is:

  http://www.imm.dtu.dk/~rrp/

He has a paper in MICCAI describing their technique:

@InProceedings{paulsen02:earmodel,
  author =       {R Paulsen and R Larsen and C Nielsen and S Laugesen and B Ersboll},
  title =        {Building and testing a statistical shape model of the human ear canal},
  booktitle =    {Medical Image Computing and Computer-Assisted Intervention},
  pages =        {373--380},
  year =         2002,
  editor =       {T Dohi and R Kikinis},
  volume =       {II},
  address =      {Tokyo, Japan},
  month =        {September},
  keywords = {Ear, Hearing aids, Medical image processing, Stereo image processing},
  abstract = {Today the design of custom in-the-ear hearing aids is based on
  personal experience and skills and not on a systematic description of the
  variation of the shape of the ear canal. In this paper it is described how
  a dense surface point distribution model of the human ear canal is built
  based on a training set of laser scanned ear impressions and a sparse set
  of anatomical landmarks placed by an expert. The landmarks are used to
  warp a template mesh onto all shapes in the training set. Using the
  vertices from the warped meshes, a 3D point distribution model is
  made. The model is used for testing for gender related differences in size
  and shape of the ear canal. (21 References).}
}

> > Yes I am capturing digital images from a digital video camera, which is
> > connected to an otoscope with an optical coupler. As the ear canal is
> > rather smooth I have the idea to project a pattern at the surface.

Yes, I think structured lighting would be easier, for a number of reasons.
The paper mentioned above uses a laser scanner, generates a point cloud and
then constructs a surface.

> > I know that it is not easy to go from this sequence of digital video
> > images in 2D to build an exact copy of the ear canal and the concha in
> > 3D. But I have a research group working for me, saying that they can get
> > at least 5000 points at the surface of the ear canal. 
> >
> > What I want to know is if you know any software, which can be used to
> > build a point cloud or a mesh model from these points.

As Luis pointed out, this is really in the domain of general Computer
Vision, specifically shape from shading and 2D->3D reconstruction.  It is
generally an under-constrained problem, as you need accurate image data as
well as accurate camera tracking.  Taking the image through an otoscope
would no doubt introduce radial distortion, so you would need a whole new
camera model to dewarp the 2D image.  Tracking will be difficult, as you
will either need physical tracking (for example, a robotic arm with shaft
encoders), or landmark tracking and registration between images.  Since the
surface of the canal is smooth and there are few unambiguous landmarks
(ie. no edges or corners!) that will be difficult also.  It's a tricky
problem... my hunch is that a structured lighting approach would be easier.

There are many papers by people such as O Faugeras and R Hartley that deal
with this sort of general problem also.

I hope this is of some help!

  :: Gavin

-- 
Gavin Baker                                      Complex Systems Group
http://www.cs.mu.oz.au/~gavinb             The University of Melbourne