VTKWidgets

Overview

The VTK widgets are undergoing a redesign. This is mainly to add functionality and clean up some inconsistencies with the existing implementation. We also have limited funding (see acknowledgements) to add new widgets in support of medical image processing (i.e., segmentation and registration).

There are four major goals of this work:

1. Support the ability to customize event bindings
2. Decouple the widget representation (geometry) from the event processing
3. Reduce redundant code; centralize common operations
4. Add new widgets in support of medical image processing

The following sections discuss eash of these goals and the proposed implementation. This page shows a variety of widget examples.

Customized Event Binding

The current VTK widgets have hardwired event bindings; e.g., a "LeftButtonPressEvent" is hardwired to execute a particular action, and this action/event binding cannot be changed. This is a problem when users desire different bindings due to personal preference, use of different interaction devices, etc. A prototype design has been created that addresses this issue and is described in the following.

Basic Concepts

The basic idea is to create a separation between VTK events and widget events (see figure). An event translator is used to translate a VTK event into a widget event. In turn, the widget event is mapped into a method invocation.

In practice this approach is overly simplified. The problem is that an event is really more than a VTK EventId, it includes various modifiers that go along with it (e.g., we'd like to bind Shift-LeftButtonPress to a particular callback). The current VTK implementation is based around only the EventId, and then leaves the callback to check for the appropriate modifiers. This poses difficulties if we want configurable event bindings, since the source code must be modified to process the modifiers separately. Hence the right way to do this is to create separate, simple callback methods that are independent of modifiers, and then insist that widget events include modifiers as part of their definition. In order to do this, the vtkWidgetEventTranslator map shown above is modified as follows. The key to the map remains the VTK event id, but the value is now a list of vtkEvents and associated widget events (see figure below). When a VTK event is received, the event id is mapped into the list, and then the list is traversed to find the matching event (the comparison takes into account the VTK event qualifiers). If found, the associated widget event is invoked. Note that the modifiers must include the notion of "ignore any modifiers" in case the user does not care whether or not the shift key is pressed at the same time the LeftButtonPress is.

Implementation Details

Here are some code snippets showing the major features of the design. Refer to the vtkSliderWidget code. Thanks to Brad for helping out here.

In the constructor of a widget, instances of vtkWidgetEventTranslator and vtkWidgetCallbackMapper are created. (In actual implementation, the vtkWidgetEventTranslator is instantiated in the superclass (vtkAbstractWidget) Once these classes are implemented, then the default event bindings are set up.

 this->EventTranslator = vtkWidgetEventTranslator::New(); //should be in superclass

 this->CallbackMapper = new vtkWidgetCallbackMapper<vtkSliderWidget>(this->EventTranslator);
 this->CallbackMapper->SetCallbackMethod(vtkCommand::LeftButtonPressEvent,
                                         vtkWidgetEvents::Select,
                                         this, vtkSliderWidget::SelectAction);

This version of SetCallbackMethod() sets things up so that modifiers to the event are ignored. Alternatively, the following method allows specification of all of the modifiers:

 this->CallbackMapper->SetCallbackMethod(vtkCommand::LeftButtonPressEvent,
                                         modifiers, keyCode, repeatCount, keySym,
                                         vtkWidgetEvents::Select,
                                         this, vtkSliderWidget::SelectAction);

Note that the vtkWidgeCallbackMapper must invoke methods on a particular class. The original design called for a templated class (templated on the widget class type). However this was discarded because either 1) the vtkWidgetCallbackMapper<T> would have to instantiated in each and every class (meaning duplicated code) or 2) very fancy template tricks would have to be used to centralize the instantiation and use of vtkWidgetCallbackMapper<T>. Instead of the fancy templated approach, a simpler approach based on static class methods is used.

Once the widget is constructed, the user can change the VTK event bindings as shown from TestSliderWidget.cxx:

 vtkSliderWidget *sliderWidget = vtkSliderWidget::New();
 sliderWidget->GetEventTranslator()->SetTranslation(vtkCommand::RightButtonPressEvent,
                                                    vtkWidgetEvent::Select);
 sliderWidget->GetEventTranslator()->SetTranslation(vtkCommand::RightButtonReleaseEvent,
                                                    vtkWidgetEvent::EndSelect);

The use of static class methods allows the code that sets up the cllbacks to be cetralized. In vtkSliderWidget.h we find the declaration of the callback methods:

 // These are the events that are handled
 static void SelectAction(vtkAbstractWidget*);
 static void EndSelectAction(vtkAbstractWidget*);
 static void MoveAction(vtkAbstractWidget*);

These are referred to in the constructor of the vtkSliderWidget:

vtkSliderWidget::vtkSliderWidget()
{
  // Set the initial state
  this->WidgetState = vtkSliderWidget::Start;

  // Assign initial values
  this->AnimationMode = vtkSliderWidget::Jump;
  this->NumberOfAnimationSteps = 24;

  // Okay, define the events
  this->CallbackMapper->SetCallbackMethod(vtkCommand::LeftButtonPressEvent,
                                          vtkWidgetEvent::Select,
                                          this, vtkSliderWidget::SelectAction);
  this->CallbackMapper->SetCallbackMethod(vtkCommand::MouseMoveEvent,
                                          vtkWidgetEvent::Move,
                                          this, vtkSliderWidget::MoveAction);
  this->CallbackMapper->SetCallbackMethod(vtkCommand::LeftButtonReleaseEvent,
                                          vtkWidgetEvent::EndSelect,
                                          this, vtkSliderWidget::EndSelectAction);
}

To set up the callbacks in the widget (i.e., add observers), the centralized code in vtkAbstractWidget::SetEnabled() is executed:

   // listen for the events found in the EventTranslator
   this->EventTranslator->AddEventsToInteractor(this->Interactor,
                                                this->EventCallbackCommand,this->Priority);

which is a very nice reduction in the code that was there previously. Removing the callbacks does not change:

   // don't listen for events any more
   this->Interactor->RemoveObserver(this->EventCallbackCommand);

Finally, the ProcessEvents() code also simplifies dramatically and is centralized in vtkAbstractWidget:

void vtkAbstractWidget::ProcessEvents(vtkObject* vtkNotUsed(object), 
                                       unsigned long vtkEvent,
                                       void* clientdata, 
                                       void* vtkNotUsed(calldata))
{
  vtkAbstractWidget* self = 
    reinterpret_cast<vtkAbstractWidget *>( clientdata );
  
  unsigned long widgetEvent = 
    self->EventTranslator->GetTranslation(vtkEvent,
                                          vtkEvent::GetModifier(self->Interactor),
                                          self->Interactor->GetKeyCode(),
                                          self->Interactor->GetRepeatCount(),
                                          self->Interactor->GetKeySym());

  if ( widgetEvent != vtkWidgetEvent::NoEvent)
    {
    self->CallbackMapper->InvokeCallback(widgetEvent);
    }
}

That's really all there is to it, with the exception that the widget callback methods should be renamed (i.e., rather than OnLeftButtonDown() the better name would be SelectAction(), etc.).

Decouple Event Processing from Widget Geometry

One of the major goals of the widget redesign was to separate widgets into two major parts:

1. One part that performs the event processing (including event translation, etc.), and
2. another part that defines a geometric representation for the widget.

From a high level, this has been implemented by creating two abstract superclasses:

1. vtkAbstractWidget and
2. vtkWidgetRepresentation

There are two major reasons for this approach:

1. Users may want to create their own look for a widget but avoid reimplementing event processing, and
2. in the parallel, distributed computing world, the user may sit on a client where events are triggered and received, but on remote, distributed computers, the capabilities for event processing may not exist. Thus the event processing must be separated from the representation.

Practically, this means that vtkAbstractWidget adds observers, processes events, invokes callbacks and so on; while vtkWidgetRepresentation is a type of vtkProp with a specialized API for communicating with the widget, as well as built-in methods for manipulating and picking the representation. (Note that while the representation is thought of as a single vtkProp, in reality it may consist of many vtkProps.) Also, while the widget knows about its representation, the representation cannot know about the widget. This insures that the representation is independent of any event processing and may be driven externally by other mechanisms (e.g. during parallel computing). To set up the relationship between the widget and its representation we use code similar to the following:

  vtkPointHandleRepresentation3D *handleRep = vtkPointHandleRepresentation3D::New();
  handleRep->SetPosition(sphereSource->GetCenter());
  handleRep->SetPlaceFactor(2.5);
  handleRep->PlaceWidget(sphereActor->GetBounds());
  handleRep->ActiveRepresentationOn();

  vtkHandleWidget *handleWidget = vtkHandleWidget::New();
  handleWidget->SetInteractor(iren);
  handleWidget->SetRepresentation(handleRep);

  vtkHandleCallback *callback = vtkHandleCallback::New();
  callback->Sphere = sphereSource;
  handleWidget->AddObserver(vtkCommand::InteractionEvent,callback);

with the key method SetRepresentation().

The key to making this works is flushing out the basic expectations that the widget has for the representation, which of course manifests in the API that the widget and representation share. While the many widgets implemented under this design demonstrate that this approaches is feasible, defining the API has been one of the major concerns of this design since it does not appear that there is a universally common API that vtkAbstractWidget and vtkWidgetRepresentation (and subclasses) can define. While in general there are common methods, different types of widgets will have to define their own unique API. Instead, vtkWidgetRepresentation specifies an API that consists of a mandatory part, and a suggested part that subclasses are free to follow (or not). The mandatory API is:

  // SetRenderer() - the renderer in which the widget is to appear must be set.
  // BuildRepresentation() - update the geometry of the widget based on its
  //                         current state.

The suggested methods are:

  // PlaceWidget() - given bounding information, place the widget inside of it. 
  //                 The current orientation of the widget is preserved, only scaling
  //                 translation is performed. 
  // StartWidgetInteraction() - generally corresponds to a initial event (e.g.,
  //                            mouse down) that starts the interaction process
  //                            with the widget.
  // WidgetInteraction() - invoked when an event causes the widget to change 
  //                       appearance.
  // ComputeInteractionState() - given (X,Y) display coordinates in a renderer, 
  //                             what is the state of the widget?
  // GetInteractionState() - return the current state of the widget. Note that the
  //                         value of "0" typically refers to "outside". The 
  //                         interaction state is strictly a function of the
  //                         representation, and the widget/represent must agree
  //                         on what they mean.
  // Highlight() - turn on or off any highlights associated with the widget.
  //               Highlights are generally turned on when the widget is selected.
  // SizeHandles() - the representation should resize any handles associated with
  //                 it. Typically the HandleSize data member is referred to.

Most widgets also define an interaction state that is specified through an enum that lists all possible states. For example, the vtkHandleRepresentation (which is driven by vtkHandleWidget, similar to vtkPointWidget) lists the following states:

  enum _InteractionState
  {
    Outside=0,
    Nearby,
    Selecting,
    Translating,
    Scaling
  };

Subclasses of vtkHandleRepresentation (vtkPointHandleRepresentation2D and vtkPointHandleRepresentation3D) share these states, so the single vtkHandleWidget can drive both.

Additionally, the representations are subclasses of vtkProp, hence they can be added to the renderer just like any other actor, etc. This means that (at a minimum) they may need to implement the following methods:

 // void GetActors()
 // void GetActors2D()
 // void GetVolumes()
 // void ReleaseGraphicsResources()
 // int RenderOverlay()
 // int RenderOpaqueGeometry()
 // RenderTranslucentGeometry()

Of course, some of these methods can be left unimplemented (for example, if a widget does not have a volume, GetVolumes() doesn't do anything).

Code Refactoring

The code base has been simplified in several ways (see previous discussions).

• ProcessEvents(), adding observers to interactor, event mapping are all implemented in the abstract superclass vtkAbstractWidget
• SetEnabled() is implemented in vtkAbstractWidget and generally suffices for most widgets (composite widgets sometimes must extent or override the method).
• An API between the widget and its representation has been generally flushed out. Common methods are implemented in the two classes vtkAbstractWidget and vtkWidgetRepresentation

New Widgets

A reference implementation has been created. The following sections describe the implementation, and offer a link to a tarball containing the source code.

Widget Description

Various widgets have been created to test out the design. Here they are first listed in alphabetical order with a brief description of what they do. Note that the widgets are grouped with their representation(s) (or geometry). Thus the number of widgets is greater than it appears in the list below: different representations result in different user experiences.

• vtkAbstractWidget - abstract superclass of all widgets; enforces the separation of widget/representation and cetralizes much of the event processing.
• vtkBorderWidget (with vtkBorderRepresentation) - a rectangular border in the overlay plane. Serves as the superclass for many subclasses (vtkTextWidget,vtkCameraWidget,vtkCaptionWidget)
• vtkCameraWidget (with vtkCameraRepresentation) - a simple widget to keyframe the camera
• vtkCaptionWidget (with vtkCaptionRepresentation) - text in a box with an arrow to a position (i.e., the anchor) in 3D space. The box and the anchor point can be manipulated separately.
• vtkCheckerboardWidget (with vtkCheckerboardRepresentation) - interacts with a vtkImageCheckerboard. Great for comparing two images.
• vtkEvent - a general representation for a vtkEvent
• vtkHandleWidget (with vtkHandleRepresentation, vtkPointHandleRepresentation2D, vtkPointHandleRepresentation3D) - manipulation a position (vtkCoordinate)
• vtkRectilinearWipeWidget (with vtkRectilinearWipeRepresentation) - similar to a vtkCheckerboardWidget, it allows interactive comparison of two images.
• vtkSliderWidget (with vtkSliderRepresentation, vtkSliderRepresentation3D) - set a scalar value with a slider (a slider is like a 2D GUI scale).
• vtkTextWidget (with vtkTextRepresentation) - size and position text. A select callback is available so you can pop up a 2D GUI to set fonts, checnge text, etc.
• vtkWidgetCallbackMapper - infrastructure for mapping events to callbacks
• vtkWidgetEvent - a list of all the internal events that widgets generate. VTK events are translated into widget events.
• vtkWidgetEventTranslator - translate VTK events into widget events.
• vtkWidgetRepresentation - abstract superclass of widget representations. It's essentially a vtkProp with some added methods to interface to the widgets (i.e., subclasses of vtkAbstractWidget).

Getting the Implementation

A tarball is available which is just a snapshot of the VTK/Widgets directory with the new code added in (as well as modified CMakeLists.txt files, etc.). You should be able to drop this code in to replace your current widgets directory (making sure to save your own stuff if you've been coding in the directory).

Note that each of these widgets/representations have a test associated with them (Widgets/Testing/Cxx).

Issues

There are several issues with this design that need addressing.

1. What should the names of the widget events be? It you look at vtkWidgetEvents.h you will see one possibility.
2. In general naming things is always an issue.
3. Mediating widget conflicts (picking, cursors). For example, if you look at something like vtkScalarBarWidget, it internally does cursor management. That is, when the mouse pointer moves over the widget the cursor changes, and when it moves outside the widget it changes to a default shape. However, when you have multiple widgets, you get conflicts on the cursor state. This gets more confusing when you consider overlapping widgets,a nd when you are picking widgets in close proximity. A mediation class, requiring widgets to register with it, is needed to resolve these conflicts. Currently I have no plans to address this in the near future due to time and funding constraints.
4. Capturing the API between a widget and its representation. How to document this? Is there a more general API?
5. SetRepresentation(rep) assumes that rep is type vtkWidgetRepresentation*. However, this means current widgets that roughly follow the separation of widget/representation (e.g., vtkScalarBarWidget) but have a method like SetScalarBarActor() take a particular actor or other type of vtkProp*, and not a representation. The question is: do we need a vtkWidgetRepresentation class? In my experience yes, since without it the corresponding widget is too intimate with the representation. For example, if you look at the current implementation of vtkScalarBarWidget it is clear that the widget assumes way too much about the API, making it hard to use this widget for anything but a vtkScalarBarActor. IMHO a good design for a widget and its representation(s) should abstract the API between them as much as possible so its easy to add new representations.

Acknowledgements

This work has been supported through the efforts of several organizations and individuals.

• The National Library of Medicine (NLM A2D2 Award, 3D Widgets for Segmentation and Registration, PI: William J. Schroeder). Dr. Terry Yoo is the Program Manager.

• Kitware, Inc. (Will Schroeder, Sebastien Barre, Brad King)

---

VTK: [Welcome | Site Map]