[Insight-developers] Interpreted language wrappers

[Brad King]

Hello everyone,

I haven't been very active on this list in a while since I have been
working on a very isolated, separate section of the project.  This is
an update to let you all know what I've been up to for the past couple
of months.

Interpreted languages are great for quick development turnaround time
as well as enhanced user interface interaction.  I've been working on
an approach to automatically generate interpreted language wrappers
(Tcl, Python, etc) around C++ classes.  There are a few implementations
available already, but each has limitations on what types of functions
and objects can be wrapped.

The primary limitation is that they all lack support for C++ templates,
which are heavily used in Insight.  Since the interpeter code must
have all the classes it wraps compiled in, it is obvious that particular
specializations of the templates must be chosen to wrap.  Parsing C++
template classes and instantiating them is extremely difficult,
especially when specializations (and partial specializations) are
involved.  I experimented for a few weeks with various existing parsers
and possibilities of implementing my own, but it soon became clear that
such a project would be an undertaking comparable in size to Insight
itself.

Here is a general overview of the approach I've taken to the problem.
There is one existing parser that already handles templates nicely: gcc.
I've implemented an extension to the C++ front end of gcc which
writes out all the declarations and template instiations of an entire
translation unit in an XML document.  (There is a strong possibility that
this will be added to gcc as a contribution permanently.)  For now,
we plan to add a stable development (2.96+) version of gcc with the
XML extension in place to the Insight CVS repository.

I've also written a parser for this XML output which stores enough
information about the C++ program to generate wrappers into a clean
internal representation.  The program reads a configuration file
(also an XML format) which controls the wrapping process, and
then generates wrappers (Tcl only for now).

Although it is not full implemented yet, the architecture of the
generated wrappers will support much of C++, including:

  - Template specializations and instantiations of functions and classes.
  - Static and non-static methods within classes.
  - Namespace-level functions (outside classes).
  - Operators (prefix, postfix, infix).
  - Overload resolution (including C++ standard conversions). 
  - Type-conversion operators and conversion by constructor.
  - Polymorphism.
  - Nested classes and namespaces (fully qualified names).
  - Custom object creation and deletion functions (like the static
    New method instead of constructors).  These can be specified
    in the wrapper generator configuration file.

This should easily be enough support to wrap everything in Insight in
interpreted languages, but the prototypes of each step in the process
are not yet ready for use.

How we get from C++ source to wrappers:

[Insight code] --> (gcc) --> [SourceXML]

  [ConfigureXML & SourceXML] --> (generateWrappers) --> [Wrapper code]

    [Wrapper code] --> (compiler) --> [Wrapper interpreter module]

Fortunately, the entire process can be automated once the configuration
XML has been written, and even that may be able to get generated
for Insight specifically.

Another reason I'm writing this email is due to a transition in my
involvement in this project.  I've had a great summer working at Kitware
on the Insight project, but must now return to RPI (where I'm an
undergrad)
to finish my 4-year degree.  I plan to continue working on Insight on a
VERY part time basis during the fall semester, and then returning to
Kitware in January.  During the fall, I will continue answering email,
but don't know whether I'll find time to continue development.  However,
Bill Hoffman, Will Schroeder, and the GE folks will take over the
development for the time being.

Regards,
-Brad