[vtk-developers] Sized types: vtkInt32Type et al

[Brad King]

I've already discussed some of this with David, but for the list's benefit:

David Gobbi wrote:
> Last night I wrote and committed a new header file for VTK, 
> vtkSizedTypes.h,
> that defines a whole new batch of types for VTK with names like 
> vtkUnsignedInt8Type
> and vtkFloat64Type.  Currently, only vtkImageReslice.cxx uses this 
> header file.  I know
> that there are several people who are interested in types like this, so 
> please take a look
> at the file and give your thoughts.

My flurry of checkins to the very top VTK headers this week has been 
driving toward this same goal.  I've added support for signed char, long 
long, unsigned long long, __int64, and unsigned __int64 to the 
vtkTemplateMacro on platforms where they are available.  This allows us 
to have at least one type representing 8, 16, 32, and 64-bit signed and 
unsigned integers on every platform we support.

> Fully integrating these new types with VTK is a huge project

It's not as big as one might think.  Now that all the native integer 
types are supported the pipeline can support processing all different 
size and signed-ness integer component data.  The rest is just providing 
platform independent names for these types for user reference.  The 
vtkClientServer wrapping used in ParaView for heterogeneous cluster 
communication has been using such names for a couple years now.

This whole new scheme will be kept in a "vtkType" 
namespace-of-convention.  Names will be of the form vtkTypeUInt32 and 
vtkTypeInt8.  The name of the type following "vtkType" will be the same 
as the name used in the XML file formats to specify types.  There will 
also be named constants such as VTK_TYPE_UINT32 that map to the 
native-type constants such as VTK_UNSIGNED_INT.

> 1) Wrapping these new types
> 
> The files vtkParse.l and vtkParse.y would have to recognize and assign 
> values to the types,

As long as all the native types are supported by the wrappers we can 
actually get pretty far without this, but it would certainly be convenient.

> 2) Defining a few new macros
> 
> My header file already contains a vtkSizedTemplateMacro for templating 
> over all the new
> types.

What David and I decided is that a "template macro" should still support 
all native types as input because it might be given any kind of array. 
In the case that it is just doing numerical operations on the data (such 
as in most filters) then as long as the implementation used to process 
the data uses the correct size/signedness then the real type used does 
not matter.  This leads to the idea of "aliasing" types of the same 
size/signedness to reduce duplicate code generated by the 
vtkTemplateMacro.  We propose a "vtkTemplateAliasMacro" that can be used 
in place of vtkTemplateMacro when such aliasing is safe.

> Several other macros are needed:
> 
> vtkSetSizedTypeMacro(name):  a macro to automatically generate methods 
> of the form
> Set##name##ToFloat32(), Set##name##ToInt32(), etc.
> 
> vtkImageSizedTypeNameMacro(type): a macro to get the name of a type, for 
> example
> vtkImageSizedTypeNameMacro(VTK_INT) would give "vtkInt32Type" the same 
> way that
> vtkImageScalarTypeNameMacro(VTK_INT) currently gives "int".
> 
> Developing these macros is far more work than most people imagine, 
> because the situation
> is complicated by the fact that certain types are not available on all 
> platforms and a certain
> amount of macro voodoo is necessary.

A vtkTypeTraits class template with specializations for each native type 
will help solve these issues behind the scenes.

> 3) Defining new array classes
> 
> New vtkDataArray-derived classes would have to be written to support the 
> new types.  For
> example, a new vtkUnsignedInt8Array class would be a subclass of 
> vtkUnsignedCharArray.
> Or to make things as easy as possible, define statements can be used 
> instead:
> #define vtkUnsignedInt8Array vtkUnsignedCharArray
> The only difficulty with using "define" is that GetClassName() will 
> return unexpected
> results.

I think the solution for this is to view the new array types as 
platform-independent names for the native array type implementations. 
This is the same view that results in the typedefs for native types 
mentioned above.

The GetClassName() method is a run-time-type-identification scheme which 
will report the native type just like the C++ RTTI typeof() operator on 
vtkTypeInt16 will report "short".  We should introduce another method 
for vtkDataArray such as "GetDataTypeName" just like we have 
"GetDataType" and "GetDataTypeSize" currently.

For creating arrays of particular size/signedness we can use the 
vtkDataArray's CreateDataArray method with the VTK_TYPE_UINT32-style 
named constants.  A string- or method-name-based signature could be 
added to make it easy to create arrays from the wrappers.

> 4) Changing a few hundred classes to use the new data types
> 
> This is definitely a big job, but not as big as people might imagine.  
> Any classes that use
> vtkTemplateMacro would have to use vtkSizedTemplateMacro instead, and 

Any class that uses vtkTemplate

> any classes
> that have a SetScalarType()-like methods would need to have 
> vtkSizedTypeMacro() added.

We may want to establish a convention here to simplify things.  In C++ 
we can use the VTK_TYPE_UINT32 and VTK_CHAR-style constants as arguments 
to such methods.  In wrapping languages we can just use strings to name 
the types:

myfilter SetOutputScalarType Int8
myfilter.SetOutputScalarType('Float64')

Then we do not need 20 overloads of functions to set the scalar type.

Comments?
-Brad