[vtkusers] Real-Time Image Warping

[Jorge Ballesteros (Motiva)]

Hi David,

First of all, thank you very uch for your advice. I have reduced the 
resolution of the transform grid and the performance has improved a bit 
(about 15 Hz) . Even though i have to set the reduction factor to 4. 
Whenever i try to use a grater factor, let's say 8x or 16x, the 
performance decreases. I got the best results with the following parameters:

 * Image resample from 512x512x100 to 256x256x100 setting the output 
extent to 256x256x5.
 * Reduction factor = 4. Grid spacing 4x and extent reduced by same factor.
 * InterpolationModeToLinear
 * GridScalarTypeToFloat

I think i will try to use GPU-based calculation, with VTKEdge package. I 
have already used VTKEdge for Volume Rendering and the results are 
great. Any hint on how to do image warping with VTKEdge?

Thanks for all,

Jorge

David Gobbi escribió:
> Hi Jorge,
>
> I notice that in vtkTransformToGrid, you are creating a grid that has
> the same resolution as the input.  That defeats the purpose of using
> vtkTransformToGrid, since it means that you are evaluating the thin
> plate spline for every voxel in the image.  You should set the
> vtkTransformToGrid spacing to 8x or even 16x the image spacing, and
> reduce its extent by the same factor.  Then, for vtkGridTransform use
> SetInterpolationModeToCubic().
>
> Also, you might see some small improvements with
> SetGridScalarTypeToFloat().  That will speed up the creation and
> evaluation of the grid transform.
>
>    David
>
>
> On Fri, Apr 9, 2010 at 4:19 AM, Jorge Ballesteros
> <jballesteros at motivando.me> wrote:
>   
>> Hi David,
>>
>> As you indicated I have rescaled the image to 256x256x98 and the performance
>> is better. For an 256x256x3 input image the warping frame rate is about
>> 25Hz.
>>
>> Even though is a big improve, i would like to keep on improving the
>> performance, but not downsampling the image. I am using
>> vtkThinPlateTransform as the the input to a vtkTransformToGrid and then
>> using a vtkGridTransform as the reslice transform to the vtkImageReslice
>> filter. I have been trying several pipelines but this is the one that
>> returns the best results. Not sure if these are the proper functions to
>> perform a 3D image warp. Am i right? Below is an extract of the code:
>>
>> vtkSmartPointer< vtkDICOMImageReader> reader =
>> vtkSmartPointer<vtkDICOMImageReader>::New();
>>   reader->SetDirectoryName(argv[1]);
>>   reader->Update();
>>
>> vtkImageData * input = reader->GetOutput(); //256x256x3
>>
>> Downsampling...
>>
>> double point[3];
>> double target[3];
>> ...
>>
>> int limit = 15;
>>
>>   vtkSmartPointer< vtkPoints > p1 = vtkSmartPointer< vtkPoints >::New();
>>   p1->InsertNextPoint(point[0]-limit, point[1], point[2]);
>>   p1->InsertNextPoint(point[0], point[1]+limit, point[2]);
>>   p1->InsertNextPoint(point[0]+limit, point[1], point[2]);
>>   p1->InsertNextPoint(point[0], point[1]+limit, point[2]);
>>   p1->InsertNextPoint(point[0]-limit, point[1]-limit, point[2]);
>>   p1->InsertNextPoint(point[0]-limit, point[1]+limit, point[2]);
>>   p1->InsertNextPoint(point[0]+limit, point[1]-limit, point[2]);
>>   p1->InsertNextPoint(point[0]+limit, point[1]+limit, point[2]);
>>   p1->InsertNextPoint(point[0], point[1], point[2]);
>>
>>   vtkSmartPointer< vtkPoints > p2 = vtkSmartPointer< vtkPoints >::New();
>>   p2->InsertNextPoint(point[0]-limit, point[1], point[2]);
>>   p2->InsertNextPoint(point[0], point[1]+limit, point[2]);
>>   p2->InsertNextPoint(point[0]+limit, point[1], point[2]);
>>   p2->InsertNextPoint(point[0], point[1]+limit, point[2]);
>>   p2->InsertNextPoint(point[0]-limit, point[1]-limit, point[2]);
>>   p2->InsertNextPoint(point[0]-limit, point[1]+limit, point[2]);
>>   p2->InsertNextPoint(point[0]+limit, point[1]-limit, point[2]);
>>   p2->InsertNextPoint(point[0]+limit, point[1]+limit, point[2]);
>>   p2->InsertNextPoint(target[0], target[1], target[2]);
>>
>>
>>   vtkSmartPointer< vtkThinPlateSplineTransform> transform = vtkSmartPointer<
>> vtkThinPlateSplineTransform>::New();
>>   transform->SetSourceLandmarks(p1);
>>   transform->SetTargetLandmarks(p2);
>>   transform->SetBasisToR();
>>
>>   //Convert the thin plate spline to a grid
>>   vtkSmartPointer<vtkTransformToGrid> transformToGrid =
>> vtkSmartPointer<vtkTransformToGrid>::New();
>>   transformToGrid->SetInput(transform);
>>   transformToGrid->SetGridExtent(-2*limit, 2*limit, -2*limit, 2*limit, 0,
>> 0);
>>   transformToGrid->SetGridOrigin(point);
>>   transformToGrid->SetGridSpacing(input->GetSpacing());
>>   transformToGrid->SetGridScalarTypeToShort();
>>
>>   vtkSmartPointer<vtkGridTransform> gridTransform =
>> vtkSmartPointer<vtkGridTransform>::New();
>>   gridTransform->SetDisplacementGrid(transformToGrid->GetOutput());
>>
>> gridTransform->SetDisplacementShift(transformToGrid->GetDisplacementShift());
>>
>> gridTransform->SetDisplacementScale(transformToGrid->GetDisplacementScale());
>>
>>   vtkSmartPointer< vtkImageReslice > reslice =
>> vtkSmartPointer<vtkImageReslice >::New();
>>   reslice->SetInput(input);
>>   reslice->WrapOn();
>>   reslice->SetResliceTransform(gridTransform);
>>   reslice->SetInterpolationModeToLinear();
>>   reslice->SetOutputSpacing(input->GetSpacing());
>>   reslice->SetOutputExtent(0, 255, 0, 255, 0, 2);
>>   reslice->Update();
>>
>> Thanks for your support,
>>
>> Jorge.
>>
>>
>> David Gobbi wrote:
>>     
>>> Hi Jorge,
>>>
>>> For real-time image warping in VTK, you would probably have to go with
>>> 256x256 images in order to achieve the 10Hz goal that you are aiming
>>> for.  For warping a 512x512x100 volume in real time, I think your only
>>> option would be to use the GPU.
>>>
>>>   David
>>>
>>>
>>> On Thu, Apr 8, 2010 at 4:38 AM, Jorge Ballesteros
>>> <jballesteros at motivando.me> wrote:
>>>
>>>       
>>>> Hello all,
>>>>
>>>> I have been trying to perform a real-time image warping with no success.
>>>>  I
>>>> have managed to warp a 512x512x100 Image but the computation time it
>>>> takes
>>>> is very high. Then I extracted a portion of the image, just 3 slices and
>>>> warped the image but still don't meet real-time requirements.
>>>>
>>>> FYI i have been following VTK Examples,
>>>> Hybrid/Testing/TestGridWarp3d.tcl,
>>>> using the vtkImageReslice class with a vtkGridTransform as the input
>>>> transform and a 512x512x3 input image, obtaining a ~10 Hz frame rate.
>>>>
>>>> Don't know if there is any other way on doing this. It would be really
>>>> helpful if you could point me on how to achieve it.
>>>>
>>>> Thanks,
>>>>
>>>> Jorge
>>>>