[vtkusers] Augmenting 2D images with 3D polydata

[Tim Soper]

I don' t think my original post caught anybody's attention (or potentially I worded it poorly), but I thought I would post my best solution to see if there were any comments on my method. In short, I wanted to take only the visible portions of a polydata and redraw it over a 2D image being rendered with the vtkImageViewer2.  This was tricky in that the portions of the polydata are occluded and secondly, because the window coordinates in my rendered polydata scene are not the same as the image coordinates in the window where I want to overlay the visible polydata.  

My solution was to copy most of the vtkSelectVisiblePoints class. This method takes as input polydata and then creates an output where all the visible points are displayed as vertices. I created a more extensive version which searches for visible points, and then copies only the cells, lines, and strips which connect those visible points. This solved the problem of collecting only the visible polydata.  Visibility is evaluated by tranforming each point in the poly data to its display position and comparing to the z-buffer at that pixel location. I added an additional option which allows the output of the points to be left in display coordinates. What I get on the output can then be simply redrawn onto my 2D image viewer which is also in display coordinates (with maybe just a scale factor and translation added to the actor). 

Overall, not too complicated, though I'm not sure why VTK doesn't just make this more extensive version of vtkSelectVisiblePoints to include all visible polydata??? 

-Tim-





----- Original Message ----
From: Tim Soper <tim_d_soper at yahoo.com>
To: vtkusers at vtk.org
Sent: Tuesday, September 30, 2008 11:00:00 AM
Subject: [vtkusers] Augmenting 2D images with 3D polydata

Hi, didn't get a response the first time, but I'm hoping someone has some advice for this:

I am having some trouble thinking of the best way to augment a 2D image with 3D polydata.

I have a medical 2D-3D registration problem where I try to match a 3D scene (in polydata) to a 2D image of the scene. Now that I have done that, I would like to overlay say a 3D path through the scene onto the 2D image (augmented reality). But I need to clip the parts of the path that are not in the scene (say the path points that are around the corner). What is the best way to filter out only visible parts of a the poly data so that I can overlay it onto the 2D image ?

The closest filter I could find is the vtkSelectPolyData. I would have to do something like render the 3D scene, collect the 3D points of the scene from the z-buffer data, then use that as input into the loop coordinates of the vtkSelectPolyData to clip only the visible parts of my path. Then overlay that onto my 2D image...?

Is there an easier way to go about this? ... 



Thanks again

Tim Soper


Research Assistant
University of Washington
Department of Bioengineering 
Box 352142



      
_______________________________________________
This is the private VTK discussion list.
Please keep messages on-topic. Check the FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
Follow this link to subscribe/unsubscribe:
http://www.vtk.org/mailman/listinfo/vtkusers