[vtkusers] vtk and python - some questions.

[Prabhu Ramachandran]

hi,

	Before I say anything else I'd really like to say this.  David
(and anyone else who was responsible for it), thanks a lot for the
existing python wrappers and all your work on it.  I really mean it.
I wouldnt have used VTK as I currently do without the present python
wrappers.  I do my VTK development purely to learn python/vtk and
Tkinter (to generate the GUI).  I don't use much of the code I
generate.  So if I have contributed to the vtk community at all it is
solely because of the python wrappers. :) So I really thank you all.
Read on...

>>>>> "David" == David Gobbi <dgobbi at irus.rri.on.ca> writes:

    >> As the python vtk users are aware, the vtk bindings for python
    >> are far from perfect.  They certainly function and one can work
    >> around some of the problems and actually do pretty nice things.
    >> However, there are a large number of things that are not ok.
    >> Primarily, the vtk python objects do not behave as proper
    >> python objects at all.

    David> The same is true for all of Python's internal types.  A
    David> python list, for example, is not a python 'object' and
    David> doesn't behave the same way as one.  There have been many
    David> discussions on comp.lang.python in regards to this issue
    David> (the 'type vs. class' issue).

	Yes, that is true but there is a UserList and a UserDict that
can be overloaded to provide what is needed?  In any case, wouldnt it
be wonderful if you could play around with new algorithms in VTK by
using python instead of c++?  I am sure this would be nice.  It would
enable a whole host of things that can be done with python+vtk.

    David> SWIG gets around this by creating a 'wrapper' class
    David> (written in python) to supplement the underlying Python
    David> type.  I really don't think this is a good idea for VTK --
    David> the python wrappers already include a huge amount of C++
    David> code... about as much Python code (in addition to the
    David> existing C++) would be neccesary to create 'proper' python
    David> objects.
 
	This is where py_cpp is so cool!  I really havent wrapped any
real code with py_cpp but I know that it does the following.  (Sorry,
I cannot give you first hand info but I havent had the time to
_really_ play with py_cpp.  I was on the boost mailing list during
py_cpp's review and have tried to contribute as much as I could during
the time.  Hopefully in the future when I have some spare time I will
try do something for the python vtk wrappers).  So py_cpp provides the
following features:

1) py_cpp uses metaclasses to wrap c++ libs to python objects.
therefore the resulting python objects are proper python objects in
that one can derive a subclass of a wrapped vtk object just as in
python.

2) py_cpp does not require one to generate _any_ native python code.
You need to generate a small amount of "boilerplate" code to actually
wrap the class.  This is similar to what is currently done with the
vtk python wrappers.

3) py_cpp takes care of the painful parts of the python object
management like reference counting, function argument parsing etc.

4) py_cpp tries not to intrude on the existing c++ code.  This is a
major design objective.  This means one does not have to change any of
the existing c++ vtk code to generate the wrappers.

5) py_cpp *can* handle overloaded methods of a class.  Overloaded
functions are also possible but require some special work.  But this
is really not too bad since most of VTK's functionality is with the
classes.

6) Templated code is no problem with py_cpp.  This doesnt matter for
VTK though.

Problems with py_cpp:

1) It does not have a parser like swig.  I am told that finding a
proper c++ parser is a really tough job.  Infact there are no
opensource ones that people seem to be aware of.  All the existing
parsers dont have some feature or the other.  However there is hope
for VTK because we already have a working parser that can be used to
generate the current wrappers.  I don't know all the details but I
think that this may be enough to wrap vtk using py_cpp.

2) The code for the wrapper generation is not automatic like it is
with swig and has to be hand generated.  But as mentioned in (1) this
may not be a problem with vtk.

3) py_cpp requires that the boost (www.boost.org) libraries be
installed, but boost is free so this really should not be a serious
issue.  Isn't it like requiring that jdk be installed? :)

    >> I believe that at one time SWIG (www.swig.org) was used to
    >> generate the wrappings.  What were the problems with swig and
    >> why is it not used any more?

    David> As someone else replied, this was never true.  Here are the
    David> reasons for not using SWIG: 1) the current release of SWIG
    David> doesn't have a complete C++ parser, and can't be made to
    David> understand the macros in vtkSetGet.h 2) because SWIG
    David> generates both C++ and Python code for the wrapper classes,
    David> it would generate more (and less efficient) wrappers than

	I agree on both points.  I have used swig myself and found
some of the lack of features rather debilitating, esp. with C++ code.

    David> the current VTK wrapper generators 3) it wouldn't make
    David> sense to use SWIG just for the Python wrappers, it is very
    David> important (from a maintenance standpoint) that the wrappers
    David> for the different languages are generated in a consistent
    David> manner, to ensure that it isn't too much of a hassle for
    David> Ken, Will et all to deal with when they have to

	If the wrappers are generated using the same parser with new
code to do the wrapping, is it not ok?  I am sure the pain will be
worth it because of the fantastic features that a proper wrapping will
offer.

    >> There are quite a few new tools that help making proper python
    >> bindings for c++ libraries.  ExtensionClass, CXX etc. come to
    >> mind.  Most notably, there is a new tool called py_cpp (the
    >> name may change) that eases wrapping c++ classes to python.
    >> py_cpp is under active development and is available here
    >> 
    >> http://people.ne.mediaone.net/abrahams/downloads/py_cpp.html
    >> 
    >> I was wondering if we could use py_cpp and modify the current
    >> wrapper scripts to use it automatically.  Would this be
    >> possible to do?  If it is, could someone in the know please
    >> provide some hints on the auto generation of wrappers?  One has
    >> to typically write very little boilerplate code to wrap the
    >> classes.  One does require the boost libraries though.

    David> See point (3) above.  Here are two questions for you: 1) If
    David> it is easy to use these tools to generate Python wrappers
    David> for VTK, then why not provide us with a demonstration?  I
    David> really would like to see better Python wrappers, myself.

	What kind of demonstration?  If you mean a demo of py_cpp then
it is available at the site mentioned above.  There is good
documentation too explaining things better than I can dream of.  I
guess you must have already seen it.  If you mean making a simple VTK
demo with py_cpp, I would very much like to but at the moment I dont
have the time.  I came across py_cpp a month or so back and have been
busy.  If you folks can wait I will try to do something by this
December - when my course work will be over...

    David> 2) If you are concerned about the quality of the existing
    David> Python wrappers, why not take the time to contribute
    David> improvements to them?  You've provided several great

	Read above and my first paragraph in this mail. :) All I want
is some details on how exactly the current wrappers are generated and
how the parsing is done.  Atleast an overall picture.  I can read the
code but I am not familiar with lexx/yacc and dont know if I will miss
something by reading the stuff in wrap/*.  I also have no idea on
writing python wrappers from scratch (i.e. I have never written a
wrapper for a c library using the C API to python).  I did take a
cursory look at the files in wrap/ after I sent my mail and also while
making the documentation for the debian packages but I really cant say
I got very far.

	So, I was hoping for some simple documentation on the wrapping
process and how the parser works.  I think this is an important thing
to have.  There is a lot of documentation for the c++ classes but I
can't find much on the parser.  The README.txt is not too informative.
If there is any documentation please send me any urls you have.  When
I get down to doing some serious wrapping I will take a look and do
what I can.

    David> suggestions in the past, but I've only had time implement a
    David> couple of them myself.

    David> You've contributed a great deal to the VTK community in the
				^^^^^^^^^^^

	Come on! Just a couple of python applications dont make you a
great contributor.  Anyway, thanks for the kind words! :)

    David> past (and please continue to do so!) and I certainly didn't
    David> write this to dampen your desire to improve the python/VTK
    David> situation!  

	Dont worry, you said nothing that dampened me.  If someone
else doesn't look at py_cpp + vtk, I will.  But I cannot say when
exactly I will start the work.  I wanted to mention the idea on the
list, so if someone was interested and had the time, maybe they could
take the plunge...

    David> But for now, I'm going to continue to focus on the existing
    David> Python wrappers until I am presented with an alternative
    David> set of operational Python/VTK wrappers.

	Hmmm.  Some simple info on the current wrapping mechanism
would help.  Would that be possible for you to write up?  Please dont
go out of your way to do it.  Take your time.  Even a month or two is
fine by me, if possible.  If you think that reading the code is all
that is required, let me know.

thanks,
prabhu