[IGSTK-Developers] Hi, question about ImageSpatialObjectRepresentation code

[David Gobbi]

Hui Zhang wrote:

> Hi, David,
>
> It is a good implementation method if the exact transform is applied
> directly to vtkImageActor. Does it mean that we should always set the
> DisplayExtent of vtkImageActor to
> (ext[0], ext[1], ext[2], ext[3], 0, 0)
> for the extract slice is a 2D image from the original input?

Yes, if we want the extent of the displayed image to be the same as the
extent of the input image, that is what we should do. With oblique
slices, though, I usually use a display extent that is larger than the
input image extent.

Another possibility is to set the reslice OutputExtent to a 3D extent,
and then use ImageActor to select a slice.
This only allows integer slice numbers:

reslice->SetOutputExtent(ext[0], ext[1], ext[2], ext[3], ext[4], ext[5]);
imageActor->SetDisplayExtent(ext[0], ext[1], ext[2], ext[3],
sliceNumber, sliceNumber);

> What I did in my former class is:
> use vtkImageReslice to extract 2D image content, and then assign the
> output to vtkTexture. Creat a vtkPlane object according to the oblique
> and slice definition, which has the same transform with vtkReslice.
> Assign the vtkTexture to that plane to display the oblique view.

Yes, that is also what I did before the vtkImageActor was added to VTK.
Really, the two methods are nearly identical, since they both use an
OpenGL texture and a plane. But vtkImageActor and vtkTexture have
different ways of handling non-power-of-two textures: vtkImageActor does
the right thing, and vtkTexture does the wrong thing:

vtkTexture: for an NxM texture, if N or M is not a power of two, the
original image is resampled so that each dimension is a power of two.
Most people (like me, and I'm guessing that you do this too) will pad
the image to a power of two with either vtkImageReslice or with
vtkImagePad so that vtkTexture does not resample it.

vtkImageActor: for an NxM texture, if N or M is not a power of two, then
the NxM image is loaded into a corner of a power-of-two texture that is
larger in size than NxM, and the texture coordinates are adjusted to map
the NxM region of the texture to the plane.

The problems with vtkTexture are as follows:
1) the resampling is computationally expensive, and results in an image
that isn't true to the original
2) if the image is padded to avoid resampling, it is still less
efficient than vtkImageActor since a power-of-two texture is loaded
instead of just a NxM subregion of a texture

> Except the oblique view, another view / imagerepresentation proposed
> by the doctors here in Georgetown Hospital is slab-MIP view, which is
> a thick or multi-slice-blend view (multi parallel slices around the
> real displayed slice). The blend function can be MIP or alpha blending
> (ray sum). These kinds of view/imagerepresentations are used to
> display the spine around the tracked needle for vertebroplasy
> experiment, and to display the tisse plane in front of the tracked
> needle for liver biopsy experiment.

> My implentation is to use vtkImageReslice to extract those
> multi-slices and write the code to perform MIP or alpha blending
> computation before assigning it to the vtkTexture.

You are in luck, I just wrote a VTK filter for one of my projects that
does multi-slice blending, projection, and MIP. It takes a slab that is
produced from vtkImageReslice, just as you describe. I will contribute
it to VTK cvs.

- David

> Just wish these would be helpful to provide a better
> view/imagerepresentation in IGSTK. I can attach some oblique and
> slab-MIP views from our current system for some demostration.
>
> Regards
> James
>
>
> ----- Original Message ----- From: "David Gobbi" <dgobbi at atamai.com>
> To: "Hui Zhang" <zhang at isis.imac.georgetown.edu>
> Cc: "'IGSTK-developers'" <igstk-developers at public.kitware.com>
> Sent: Monday, November 14, 2005 7:00 PM
> Subject: Re: [IGSTK-Developers] Hi, question about
> ImageSpatialObjectRepresentation code
>
>
>> Hi James,
>>
>> You're right, vtkImageReslice can be used for orthogonal and oblique
>> slices, but even if ImageSpatialObjectRepresentation uses
>> vtkImageReslice to extract the slice, it should still use vtkImageActor
>> to display it (but vtkImageActor would not be used to select the
>> orientation, and probably not the slice number either).
>>
>> The trick is that whatever transform is used as the ResliceTransform for
>> vtkImageReslice, the exact same transform also has to be used as the
>> UserTransform for the vtkImageActor. That ensures that all of the pixels
>> in the resliced image are displayed at the correct position in 3D space.
>>
>> It is also necessary to make sure that the OutputExtent of
>> vtkImageReslice and the DisplayExtent of vtkImageActor are matched to
>> one another.
>>
>> And here is where I start rambling...
>>
>> Choosing the pixel spacing for the oblique slice is difficult if the
>> original voxel spacing of the image is not isotropic. I have used two
>> different ways to choose the pixel spacing:
>>
>> 1) Use the direction cosines of the oblique slice to calculate an
>> "oblique" pixel spacing based on a weighted combination of the original
>> x, y, and z spacings
>>
>> 2) My favorite method: if the View is a parallel projection (like most
>> 2D views) then it is possible to calculate exactly how many millimetres,
>> in patient coordinates, there are per screen pixel on the monitor. Then
>> the OutputExtent, OutputSpacing and OutputOrigin of vtkImageReslice can
>> be set so that each pixel in the resliced image maps to a screen pixel
>> in the View2D.
>>
>> I prefer (2) because it uses vtkImageReslice to combine the reslicing
>> and the image zoom into one trilinear interpolation step. In method (1),
>> vtkImageReslice performs trilinear interpolation for the oblique
>> reslicing, and then vtkImageActor applies a bilinear interpolation step
>> to achieve the zoom. Because two interpolations steps are done, method
>> (1) provides an image that is less true to the original.
>>
>> - David
>>
>>
>>
>>
>> Hui Zhang wrote:
>>
>>> Hi,
>>>
>>> I check the current ImageSpatialObject /
>>> ImageSpatialObjectRepresentation code from IGSTK and find it is easy
>>> to use this class to display the 2D slice image from Axial /
>>> Sagittal / Coronal view. The code use the vtkImageActor to set the
>>> orientation and slice number, as below:
>>>
>>> switch( m_Orientation )
>>> {
>>> case Axial:
>>> m_ImageActor->SetDisplayExtent( ext[0], ext[1], ext[2], ext[3],
>>> m_SliceNumber, m_SliceNumber );
>>> break;
>>> case Sagittal:
>>> m_ImageActor->SetDisplayExtent( m_SliceNumber, m_SliceNumber,
>>> ext[2], ext[3], ext[4], ext[5] );
>>> break;
>>> case Coronal:
>>> m_ImageActor->SetDisplayExtent( ext[0], ext[1], m_SliceNumber,
>>> m_SliceNumber, ext[4], ext[5] );
>>> break;
>>> }
>>>
>>> My question is, through this category, we can't display any oblique
>>> image slice and integrate them into the same class. If the surgeons
>>> need get oblique view along the planned path or tracked tool, or
>>> even slab-MIP view, the program can't make any use of the existed
>>> ImageSpatialObjectRepresentation class. That means, we'll have two
>>> different ImageRepresenation classes, one for the fixed-axis, and
>>> one for the oblique axis. What we did before in our current
>>> navigation system is, we use vtkImageReslice instead of
>>> vtkImageActor, and made the orthogonal image representations (Axial
>>> / Coronal / Sagittal) are three specific oblique views. So, in my
>>> question, can we implement with vtkImageReslice in
>>> ImageSpatialObjectRepresentation class, instead of vtkImageActor?
>>> And from this implementation, the orthogonal image representation
>>> can be easily a derived class or just three specific settings.
>>>
>>> Regards,
>>>
>>> ---------------------------------------------------------------------------------------------
>>>
>>> Hui (James) Zhang, Ph.D.
>>>
>>> Imaging Science and Information Systems (ISIS) Center
>>> Department of Radiology
>>> Georgetown University Medical Center
>>> 2115 Wisconsin Avenue, Suite 603
>>> Washington, DC, 20007
>>>
>>> Work phone: 202-687-2902
>>> Work fax: 202-784-3479
>>> Cell phone: 240-476-9361
>>> MSN: zh_skyfox at hotmail.com
>>> ----------------------------------------------------------------------------------------------
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> _______________________________________________
>>> IGSTK-Developers mailing list
>>> IGSTK-Developers at public.kitware.com
>>> http://public.kitware.com/cgi-bin/mailman/listinfo/igstk-developers
>>>
>>>
>>
>>
>>
>
>
>