[vtkusers] vtk51 cvs memory corruption in vtkInteractorObserver derived classes

[Andrew J. Dolgert]

Hi,

I'm running a cvs snapshot from 19 March under Visual Studio 2005 on
Windows XP. I found a problem in how vtkInteractorObserver receives
callbacks that leads to memory corruption. I think most compilers don't
notice it, but Microsoft Visual Studio does throw exceptions under
certain circumstances.

In its constructor, vtkInteractorObserver initializes its callback
object as follows:

vtkInteractorObserver::vtkInteractorObserver()
{
  this->Enabled = 0;
  this->Interactor = NULL;
  this->EventCallbackCommand = vtkCallbackCommand::New();
  this->EventCallbackCommand->SetClientData(this);

  <more stuff>
}

Then a derived class, like vtkBoxWidget, registers its callback in its
constructor:

vtkBoxWidget::vtkBoxWidget()
{
  this->State = vtkBoxWidget::Start;
  this->EventCallbackCommand->SetCallback(vtkBoxWidget::ProcessEvents);

  <more stuff>
}

In ProcessEvents, the pointer to vtkInteractorObserver (its "this"), is
cast to a vtkBoxWidget. That's a problem.

  vtkBoxWidget* self = reinterpret_cast<vtkBoxWidget*>(clientdata);
  self->OnLeftButtonDown();

The "this" pointer of a base class is not guaranteed to be the same as
that of a derived class and often is shifted by a few bytes. Visual
Studio is throwing a System.AccessViolationException. If the derived
class were to call SetClientData(), then all would be well.

Thanks,
Drew Dolgert