VTK/Python Wrapper Enhancement

This project will improve the python wrappers bit-by-bit, with the goal of making the Python interface as close as possible to the original C++ interface. When each piece is finished, it will be marked as "done".

The original author of this document is David Gobbi. He can be reached on the VTK developers mailing list.

Improvements to Python wrappers

• Allow hierarchies of special types (i.e. non-vtkObjectBase classes)
• Enums and other constants (done)
• Operator support (partly done)
• Wrapping of templated types - will always be limited to selected types, but can still be very useful
• Wrapping of default-value arguments (done)
• Wrapping of pointers args - requires a better hinting system
• Wrapping of reference args and returning values in them (done)
• Wrapping of multi-dimensional arrays (done)
• Handle SetTuple/GetTuple methods of vtkDataArray (done)
• Wrap vtkCommand as a special type

Hierarchies of special types in Python wrappers

If each non-vtkObjectBase special type had its own PyTypeObject struct (generated by vtkWrapPython.c) then:

• These types could have a hierarchy via python's subclass system
• Type-specific protocols (number, sequence, buffer, etc) could be supported, this would require proper parsing of operators

Also: see VTK/WrapHierarchy for a tool that provides the entire hierarchy to the wrappers at compile-time. Note that the WrapHierarchy tool is of critical importance... without it, the wrappers cannot tell if a "vtkSomething *" parameter is a pointer to a vtkObjectBase object or to a special VTK type.

Enums and constants (done as of July 31, 2010)

The new vtkParse will parse enums and #define constants.

• The FileInfo struct must be expanded to hold these constants
• Constant values should be stored as strings to simplify typing*
• The wrappers will have to automatically add the class scope to enum constants.

* The strings can be written literally into the wrapper .cxx files where they will be evaluated as the correct type.

Operator support

The new vtkParse provides information about operator methods for the VTK classes. These operator methods can be mirrored in Python by:

1. defining the appropriate "protocols" for special type objects
2. defining the proper methods e.g. __setitem__() for vtkObjectBase objects

Each VTK special type will have its own python "type" object, and can thus support its own set of protocols that will be inherited by "subtypes". All vtkObjectBase objects have the same python "type" object so protocols cannot be used, but underscore methods can potentially be used.

There is not any support in vtkParse for functions that are defined outside of classes. The one exception to this is the following:

ostream &operator<<(ostream &, someObject);

The above method is used to implement the str(), repr(), and "print" methods of special objects.

Templated type handling in Python

Should be made to look similar to numpy, e.g. vtkValue(1, 'f') would create a vtkValue<float>. To python, the templated type would look like a variadic type. It would be necessary to change vtkParse so that it recognized templates.

Default value arguments (done as of Sept 17, 2010)

Default argument value support would require the following:

1. vtkParse must store the default value in the FunctionInfo struct as a string*
2. vtkWrapPython.c must use these default values to initialize parameter values
3. vtkWrapPython.c must place a bar "|" before the default args in the ParseTuple format string
4. some other small changes would be needed

* the default value must be stored as a string to accommodate all types and to accommodate simple mathematical expressions, e.g. the default might be SomeMethod(int param = VTK_CONST1 - VTK_CONST2). The string "VTK_CONST1 - VTK_CONST2" can be dropped directly into the wrapper CXX code where it will be evaluated.

Pointer arg wrapping

Pointer arguments (as opposed to array arguments) are used for the following in VTK:

1. passing data arrays, e.g. vtkIntArray::SetArray(int *data, vtkIdType size, int save)
2. 'tuples' where the tuple size can change, e.g. vtkDataArray::SetTuple(double *tuple)
3. return slots, e.g. vtkVariant::ToInt(bool *valid)

For (1) and (2), there is an analogous situations for return values:

1. int *vtkIntArray::GetPointer(vtkIdType offset)

The closest thing that Python has to a "pointer" is its buffer objects, such as array, string, and numeric array. The problem is that a python buffer always requires a size argument, but C++ rarely provides any hints about the size of the data object that a pointer is pointing to. Some heuristics would have to be applied:

1. the wrappers can be made to look for vtkDataArray and properly handle their pointer methods
2. other methods will need some sort of hinting.

The "count" for pointer args should be hinted so that they can be properly wrapped. E.g.

1. vtkVariant::ToInt(bool *vtkSingleValue(valid))
2. vtkVariant::ToInt(bool *vtkOptionalSingleValue(valid)) - can be safely set to NULL
3. vtkDataArray::SetTuple(double *vtkMultiValue(tuple, ->GetNumberOfComponents()))

In the latter, the name of the method to get the count is supplied in the hint. Recognizing these macros in vtkParse would be easy. Unfortunately, they make the C++ code very ugly.

Reference arg wrapping (done as of Sept 17, 2010)

This is trivial to add, only a few lines would have to be added to vtkWrapPython. For the reference arg, the user would have to pass a container object that supported both the sequence protocol and the number protocol. A new "mutable" object was created for this purpose.

Wrapping of multi-dimensional arrays (done as of Sept 17, 2010)

Python can unpack nested sequences, so reading multi-array args is easy. Writing back to them is a bit more complicated.

GetTuple/SetTuple (done as of Aug 6, 2010)

The wrappers should make a "special case" for vtkDataArray and wrap the GetTuple/SetTuple methods using the knowledge that the tuple size is equal to the number of components. The same can be done for the subclasses, with GetTupleValue/SetTupleValue. This is a change that could also be easily done for Tcl and Java. (done as of Aug 6, 2010 for Python wrappers).

Wrap vtkCommand as a special type

Right now, vtkCommand() cannot be instantiated from Python. The wrapped AddObserver method takes a python function as an argument, and then secretly convertes it into a vtkPythonCommand. This means that none of the vtkCommand methods are available from Python. such as

• SetAbortFlag()/GetAbortFlag()
• SetPassiveObserver()

I have done some work to remedy this, but the work is not yet complete. So far, I have:

1. changed the CMake files so that vtkCommand is wrapped (but it is abstract)
2. made it possible to subclass vtkCommand in python (but the subclasses are abstract)

To make everything work, vtkCommand subclasses in python must actually be subclassed from vtkPythonCommand (which is concrete), and vtkPythonCommand must provide virtual function hooks so that Execute can be overridden as a virtual method. This is only possible if vtkPythonCommand is provided with a "PyObject *" slot for its pythonic other half, so that when Execute is called it can search the python dict to see if the method has been overridden.