[vtk-developers] vtkMessage, vtkMessageQueue, and threads

[Michael Halle]

In the interest of promoting communication about developing projects,
I'd like to pass on a summary of something I'm working on.  One of the
hardest software problems of real-world visualization systems is how
to get data from disparate data sources into the program.  I'm
specifically talking about data from probes and other input devices,
but it broadens to an entire class of data.  I've been putting
together some classes to make this process easier that should mesh
with a variety of input loops in any language.

Motivation

The kind of data that I'm most interested in is different than what
VTK handles well right now.  I'd characterize it as small and possibly
dynamic data.  For instance, a probe might generate a stream of
coordinates and orientations.  A tablet might also include pressure.
A data aquisition device might send a bunch of samples of some kind.
One device might send different kinds of events, and one program might
get similar data from different sources.

It would be great to visualize these kind of streams, but it's painful
to get that kind of data into vtk.  You could write a class from
scratch in C++, but without some structure every program that uses
your data is different.  It's difficult to process that data in your
event loop, of more specifically to get your class to work with events
loops written in an arbitrary scripting language.  You could also
write your data I/O in, say, Tcl, but you again lose the portability.
Many event sources are most simply implemented using a multithreaded
model, but that only complicates the design.

Messages, Pipes and Queues

Here's what I've come up with to fix this problem.  I implemented a
data type called vtkMessage.  A message is made up of typed fields
with unique string names.  A field can be an int, float, double,
string, or opaque (byte string) type.  Message fields can be
introspected (I look forward to the use of vtkHashTable!).  A message,
then, is the building block for describing simple, dynamic data.

Modules communicate with each other by placing messages on message
pipes.  A message pipe is essentially just an abstract class: if
provides the Push and Pop methods.  Currently, the only subclass of a
message pipe is a vtkMessageQueue.  A message queue is a thread-safe
queue of vtkMessages (duh) through which threads of control can
communicate.  Message queues can also provide a compressor method that
will intelligently drop extraneous events (for instance, get rid
of extra pointer motion events but keep button click events).  

The Pop and Push methods of pipes and queues have both non-blocking
and blocking modes.  To implement blocking, I added a
vtkConditionVariable class.  A condition variable augments a mutex and
allows a thread to sleep/block until a particular condition is met
(like a message being pushed onto a queue by another thread).  There's
a simple pthreads implementation of condition variables, and I've
found a native Win32 implementation, but I'd need some help in testing
it since I don't use that platform.

Computational and Communicating Elements

With message pipes and queues, it becomes possible to create an
event-driven pipeline that's different and complementary to the
standard VTK pipeline.  Elements of the pipeline are threads that read
and write from message queues as well as outside data sources.  The
threads can poll or block waiting for input.  The thread-safeness of
the message pipes/queues isolate the threads and make programming them
much simpler than the arbitrary case.  It works very much like a unix
pipeline, with the added benefit of structured messages and the
ability to build bidirectional elements.  Unlike vtk, multithreading
across modules is natural.

Examples

What kind of things can plug into the pipeline?  I've implemented two
samples of these types of modules (I call them messengers).  The first
is a module that reads from a graphics tablet in a non-portable,
UNIX-y kind of way.  The reader is a thread.  Every tablet event (x,
y, pressure, modifier) gets turned into a message by adding four
fields.  The message is then pushed onto an event queue, and the
thread goes back to inspecting the device file descriptor.  Meanwhile,
the main vtk thread of control is doing its normal rendering.  I run
an idle task (implemented in vtk) that checks the other end of the
message queue every once in a while to see if there's a message.  If
there is one, if extracts the field values from the message and does
something interesting with it.  The application is responsible for
making the queue mesh with the local event loop.

My other "messenger" module is an interface to a neat piece of
software called Elvin (http://elvin.dstc.com/).  An elvin server
routes messages that looks very much like vtkMessages from producers
to consumers that have expressed interest in them.  It's possible to
write an elvin message producer in about ten lines of code. That means
that instead of tying a data probe directly to vtk, we can more easily
tie it to elvin and have vtk just handle the probe's elvin message.
All of a sudden, vtk can talk to lots of things just by pushing
and popping messages to the elvin messenger.

With the flexibility of message queues, it was also pretty simple
to tie the tablet messenger to the elvin messenger and produce
an networked tablet device.  

Other technical issues: threads

Blocking threads are not dealt with well in vtk right now.
vtkMultiThreader requires threads to poll a flag to know if they
should quit or not, but if a thread is waiting for a condition
variable there's no way to poll the flag.  My solution was to create a
full-blown vtkThread class.  It follows the POSIX semantics but is
wrapped nicely in C++.  Subclasses implement a Run() method.  The run
method is kicked off by calling Start().  A thread is can be
Cancel()ed by another thread (usually the main thread).  A thread can
disable cancellation or defer it until it called TestCancel().
Cancellation breaks threads out of condition variable blocks.  Upon
cancellation, a thread's Cleanup() method automatically gets called.

I think it would be difficult to implement this class natively
for Win32.  However, there is a Win32-Pthreads library that would
allow the vtkThread method to be used unmodified.  The new thread
class, while I think it very nice, is only necessary because of
the blocking behavior.  If you're willing to never terminate
your threads, or you can have them poll a flag, you don't need
cancellation.  What's the best way to accommodate this restriction
in VTK?

MutexLock vs. CriticalSection

I wrote ConditionVariable to be subclass of MutexLock. Then, I
happened to come across vtkCriticalSection.  The only difference seems
to be in the Win32 implementation.  Does the limited use of locking
in vtk require both of these classes to exist independently?  It
seems they could be rolled into one pretty easily, even if
both kinds of behavior are important.  In fact, I think 
MutexLock, CriticalSection, and ConditionVariable could really
all be the same class: all ConditionVariable provides are 
extra Signal() and Wait() methods and functionality.

I'd be interested in comments and suggestions about the concept and
design and how it fits together with other developments (for instance,
the Observer/Widget work.)


							--Mike