[vtkusers] ImageViewer2 display/picking problem

[Mike Dresser, MD, PhD]

Hello all.

I apologize in advance for the length of this posting but there has been a
lot of interest in and trouble with picking.  I ran into a problem with
picking, chased it and found a problem with display, and came up with a
solution for my particular application that may not be general but may be
useful for some, so I'm including some details here.  There very well may be
a better/simpler way to do this.  If so, I'd appreciate hearing it.

I'm using WIN2K, Visual C++ 6, the VTK nightly from 4/21/2004 and
ImageViewer2 to visualize single-plane fluorescence microscopy images.  I
ran into the problem that the edge pixels are 1/2 the width of the other
pixels, which is most easily seen by using
viewer->GetImageActor()->SetInterpolate(0) and zooming in.  I tracked the
origins of this to the setup of the world and texture coordinates in
vtkOpenGLImageActor.  The problem begins when, for example, an image that is
16 pixels in width is rendered into coordinates extending from 0 to 15
(starting on line 118).  Information from all the pixels are actually
displayed, because the texture coordinates are adjusted by 0.5 (lines173 and
212), but there's only "room" for 1/2 width display of the boundary values.
This also makes picking less than straightforward, as the edges of
individual pixels don't lie at integer coordinates.  The picking can be
sorted out without changing vtkOpenGLImageActor, but I don't see a simple
way to fix the display post-hoc.

So, here's what works for me, with fairly limited testing, so far.  It
requires modifying vtkOpenGLImageActor, and there are side-effects, but at
least for my application, they are side effects that can be "fixed."

In vtkOpenGLImageActor:

Replace (starting line 117)

// compute the world coordinates
this->Coords[0] = this->DisplayExtent[0]*spacing[0] + origin[0];
this->Coords[1] = this->DisplayExtent[2]*spacing[1] + origin[1];
this->Coords[2] = this->DisplayExtent[4]*spacing[2] + origin[2];
this->Coords[3] = this->DisplayExtent[1]*spacing[0] + origin[0];
this->Coords[4] = 
    this->DisplayExtent[2 + (xdim == 1)]*spacing[1] + origin[1];
this->Coords[5] = this->DisplayExtent[4]*spacing[2] + origin[2];
this->Coords[6] = this->DisplayExtent[1]*spacing[0] + origin[0];
this->Coords[7] = this->DisplayExtent[3]*spacing[1] + origin[1];
this->Coords[8] = this->DisplayExtent[5]*spacing[2] + origin[2];
this->Coords[9] = this->DisplayExtent[0]*spacing[0] + origin[0];
this->Coords[10] = 
    this->DisplayExtent[2 + (ydim == 1)]*spacing[1] + origin[1];
this->Coords[11] = this->DisplayExtent[5]*spacing[2] + origin[2];
  
with

// compute the world coordinates  
this->Coords[0] = this->DisplayExtent[0]*spacing[0] + origin[0];
this->Coords[1] = this->DisplayExtent[2]*spacing[1] + origin[1];
this->Coords[2] = this->DisplayExtent[4]*spacing[2] + origin[2];
if (xdim == 0)  {this->Coords[3] = (this->DisplayExtent[1] + 1)*spacing[0] +
origin[0];}
  else {this->Coords[3] = this->DisplayExtent[1]*spacing[0] + origin[0];}
if (xdim == 1)  {this->Coords[4] = (this->DisplayExtent[3] + 1)*spacing[1] +
origin[1];}
  else {this->Coords[4] = this->DisplayExtent[2]*spacing[1] + origin[1];}
this->Coords[5] = this->DisplayExtent[4]*spacing[2] + origin[2];
if (xdim == 0)  {this->Coords[6] = (this->DisplayExtent[1] + 1)*spacing[0] +
origin[0];}
  else {this->Coords[6] = this->DisplayExtent[1]*spacing[0] + origin[0];}
if (xdim == 1 || ydim == 1)  {this->Coords[7] = (this->DisplayExtent[3] +
1)*spacing[1] + origin[1];}
  else {this->Coords[7] = this->DisplayExtent[3]*spacing[1] + origin[1];}
if (ydim == 2)  {this->Coords[8] = (this->DisplayExtent[5] + 1)*spacing[2] +
origin[2];}
  else {this->Coords[8] = this->DisplayExtent[5]*spacing[2] + origin[2];}
this->Coords[9] = this->DisplayExtent[0]*spacing[0] + origin[0];
if (ydim == 1)  {this->Coords[10] = (this->DisplayExtent[3] + 1)*spacing[1]
+ origin[1];}
  else {this->Coords[10] = this->DisplayExtent[2]*spacing[1] + origin[1];}
if (ydim == 2)  {this->Coords[11] = (this->DisplayExtent[5] + 1)*spacing[2]
+ origin[2];}
  else {this->Coords[11] = this->DisplayExtent[5]*spacing[2] + origin[2];}

Replace (starting line 173)

this->TCoords[0] = (this->DisplayExtent[xdim*2] - ext[xdim*2] + 0.5)/xsize;
this->TCoords[1] = 0.5/ysize;  
this->TCoords[2] = (this->DisplayExtent[xdim*2+1] - ext[xdim*2] +
0.5)/xsize;
this->TCoords[3] = this->TCoords[1];  
this->TCoords[4] = this->TCoords[2];
this->TCoords[5] = 1.0 - 0.5/ysize;  
this->TCoords[6] = this->TCoords[0];
this->TCoords[7] = this->TCoords[5];

with

this->TCoords[0] = 0.0;
this->TCoords[1] = 0.0;  
this->TCoords[2] = 1.0;
this->TCoords[3] = 0.0;  
this->TCoords[4] = 1.0;
this->TCoords[5] = 1.0;  
this->TCoords[6] = 0.0;
this->TCoords[7] = 1.0;

	Replace   (starting line 212)

this->TCoords[0] = 0.5/xsize;
this->TCoords[1] = 0.5/ysize;  
this->TCoords[2] = (this->DisplayExtent[xdim*2+1] -
this->DisplayExtent[xdim*2] + 0.5)/xsize;
this->TCoords[3] = this->TCoords[1];  
this->TCoords[4] = this->TCoords[2];
this->TCoords[5] = (this->DisplayExtent[ydim*2+1] -
this->DisplayExtent[ydim*2] + 0.5)/ysize;  
this->TCoords[6] = this->TCoords[0];
this->TCoords[7] = this->TCoords[5];  

	with 

this->TCoords[0] = 0.0;
this->TCoords[1] = 0.0;
this->TCoords[2] = (this->DisplayExtent[xdim*2+1] -
this->DisplayExtent[xdim*2] + 1.0)/xsize;
this->TCoords[3] = this->TCoords[1];  
this->TCoords[4] = this->TCoords[2];
this->TCoords[5] = (this->DisplayExtent[ydim*2+1] -
this->DisplayExtent[ydim*2] + 1.0)/ysize;
this->TCoords[6] = this->TCoords[0];
this->TCoords[7] = this->TCoords[5];

The above three changes put pixel edges on integer coordinates and display
all pixels at equal size.  Of course, you still have to keep up with spacing
and origin when you pick.

One side effect is that the display is no longer centered when it first
comes up.  I've normalized the spacing along the axes by the spacing along Y
before the following block, so it doesn't need "correction" going into this:

camera = viewer->GetRenderer()->GetActiveCamera();	
double foc[3];
double pos[3];
camera->GetFocalPoint(foc);
camera->GetPosition(pos);
 //Y okay as   is (==1) and Z is superfluous
camera->SetFocalPoint( ( foc[0] + 0.5 ) * m_dxSpacing, foc[1] + 0.5,
foc[2]);
camera->SetPosition( ( pos[0] + 0.5 ) * m_dxSpacing, pos[1] + 0.5, pos[2]); 


To pick, I trap viaPreTranslateMessage (MSG* pMsg) and do the following:

if (pMsg->message == WM_MOUSEMOVE)  {

 //convert lParam to X and Y points that can be picked
 int X = MAKEPOINTS(pMsg->lParam).x;
 int Y = MAKEPOINTS(pMsg->lParam).y;
 RECT rcViewClient;		
 GetClientRect(&rcViewClient);
 //the image data has been sent through vtkImageFlip
 Y = rcViewClient.bottom - Y - 1;

 vtkPropPicker* picker = vtkPropPicker::SafeDownCast(inter->GetPicker());
 //haven't need to do: picker->SetTolerance(0.0001);
 picker->Pick(X,Y, 0.0, viewer->GetRenderer());
 double* pos = picker->GetPickPosition();
 double intensity;
 int xPos, yPos;

 //correct for offset of origin
 pos[0] -= m_dOriginX;
 pos[1] -= m_dOriginY;

 //find intensity associated with this voxel
 if (pos[0] > 0.0 && pos[1] > 0.0)  {

  //works with modified vtkOpenGLImageActor 20040518
  //to correct for the normalization of Y spacing to 1 in OnInitialUpdate
  //above, did "m_dxySpacingRatio = m_dySpacing / m_dxSpacing;"
  xPos = (int)(pos[0] * m_dxySpacingRatio);
  yPos = (int)(pos[1]);

  //there can be a little "slop" at the edges, so correct for this
  //Note that this needs to be corrected for XY, XZ or YZ plane dispExt[?]
if ever
  //using this routine for 2D texture maps from 3D data
  int dispExt[6];
  viewer->GetImageActor()->GetDisplayExtent(dispExt);
  //if showing XY plane
  if ( m_nMaxZ == 1)  { 
   xPos = ( xPos < dispExt[0] ? dispExt[0] : xPos );
   xPos = ( xPos > dispExt[1] ? dispExt[1] : xPos );
   yPos = ( yPos < dispExt[2] ? dispExt[2] : yPos );
   yPos = ( yPos > dispExt[3] ? dispExt[3] : yPos );
  }

//untested for showing XZ or YZ planes, so I've omitted the code

//flipper is vtkImageFlip, which inverted the image along Y
intensity = flipper->GetOutput()->GetScalarComponentAsDouble (xPos, yPos, 0,
0);

//for display on window message bar
CMainFrame *pFrame = (CMainFrame*)AfxGetApp()->m_pMainWnd;
pFrame->m_nX = xPos + 1;
pFrame->m_nY = yPos + 1;
pFrame->m_dIntensity = intensity;
 ((CStatusBar*)pFrame->GetMessageBar())->OnUpdateCmdUI(pFrame,FALSE);
}


That's ~all of it except for housekeeping as regards spacing and origin
changes.  Another thing to take note of is that cropping, e. g.:

viewer->GetImageActor()->SetDisplayExtent(5,10,10,15,0,0); 

results in a 6 X 6 pixel display, which some might find unexpected (and
requires recentering the displayed image, if desired).  I don't know if
changes to vtkOpenGLImageActor are warranted, or might break lots of other
things, so buyer beware (unless Kitware does it).

Mike Dresser
405-271-7682
Oklahoma Medical Research Foundation
825 NE 13th St.
Oklahoma City, OK 73104