[vtkusers] Real-Time Image Warping

[David Gobbi]

Hi Jorge,

I notice that in vtkTransformToGrid, you are creating a grid that has
the same resolution as the input.  That defeats the purpose of using
vtkTransformToGrid, since it means that you are evaluating the thin
plate spline for every voxel in the image.  You should set the
vtkTransformToGrid spacing to 8x or even 16x the image spacing, and
reduce its extent by the same factor.  Then, for vtkGridTransform use
SetInterpolationModeToCubic().

Also, you might see some small improvements with
SetGridScalarTypeToFloat().  That will speed up the creation and
evaluation of the grid transform.

   David


On Fri, Apr 9, 2010 at 4:19 AM, Jorge Ballesteros
<jballesteros at motivando.me> wrote:
> Hi David,
>
> As you indicated I have rescaled the image to 256x256x98 and the performance
> is better. For an 256x256x3 input image the warping frame rate is about
> 25Hz.
>
> Even though is a big improve, i would like to keep on improving the
> performance, but not downsampling the image. I am using
> vtkThinPlateTransform as the the input to a vtkTransformToGrid and then
> using a vtkGridTransform as the reslice transform to the vtkImageReslice
> filter. I have been trying several pipelines but this is the one that
> returns the best results. Not sure if these are the proper functions to
> perform a 3D image warp. Am i right? Below is an extract of the code:
>
> vtkSmartPointer< vtkDICOMImageReader> reader =
> vtkSmartPointer<vtkDICOMImageReader>::New();
>   reader->SetDirectoryName(argv[1]);
>   reader->Update();
>
> vtkImageData * input = reader->GetOutput(); //256x256x3
>
> Downsampling...
>
> double point[3];
> double target[3];
> ...
>
> int limit = 15;
>
>   vtkSmartPointer< vtkPoints > p1 = vtkSmartPointer< vtkPoints >::New();
>   p1->InsertNextPoint(point[0]-limit, point[1], point[2]);
>   p1->InsertNextPoint(point[0], point[1]+limit, point[2]);
>   p1->InsertNextPoint(point[0]+limit, point[1], point[2]);
>   p1->InsertNextPoint(point[0], point[1]+limit, point[2]);
>   p1->InsertNextPoint(point[0]-limit, point[1]-limit, point[2]);
>   p1->InsertNextPoint(point[0]-limit, point[1]+limit, point[2]);
>   p1->InsertNextPoint(point[0]+limit, point[1]-limit, point[2]);
>   p1->InsertNextPoint(point[0]+limit, point[1]+limit, point[2]);
>   p1->InsertNextPoint(point[0], point[1], point[2]);
>
>   vtkSmartPointer< vtkPoints > p2 = vtkSmartPointer< vtkPoints >::New();
>   p2->InsertNextPoint(point[0]-limit, point[1], point[2]);
>   p2->InsertNextPoint(point[0], point[1]+limit, point[2]);
>   p2->InsertNextPoint(point[0]+limit, point[1], point[2]);
>   p2->InsertNextPoint(point[0], point[1]+limit, point[2]);
>   p2->InsertNextPoint(point[0]-limit, point[1]-limit, point[2]);
>   p2->InsertNextPoint(point[0]-limit, point[1]+limit, point[2]);
>   p2->InsertNextPoint(point[0]+limit, point[1]-limit, point[2]);
>   p2->InsertNextPoint(point[0]+limit, point[1]+limit, point[2]);
>   p2->InsertNextPoint(target[0], target[1], target[2]);
>
>
>   vtkSmartPointer< vtkThinPlateSplineTransform> transform = vtkSmartPointer<
> vtkThinPlateSplineTransform>::New();
>   transform->SetSourceLandmarks(p1);
>   transform->SetTargetLandmarks(p2);
>   transform->SetBasisToR();
>
>   //Convert the thin plate spline to a grid
>   vtkSmartPointer<vtkTransformToGrid> transformToGrid =
> vtkSmartPointer<vtkTransformToGrid>::New();
>   transformToGrid->SetInput(transform);
>   transformToGrid->SetGridExtent(-2*limit, 2*limit, -2*limit, 2*limit, 0,
> 0);
>   transformToGrid->SetGridOrigin(point);
>   transformToGrid->SetGridSpacing(input->GetSpacing());
>   transformToGrid->SetGridScalarTypeToShort();
>
>   vtkSmartPointer<vtkGridTransform> gridTransform =
> vtkSmartPointer<vtkGridTransform>::New();
>   gridTransform->SetDisplacementGrid(transformToGrid->GetOutput());
>
> gridTransform->SetDisplacementShift(transformToGrid->GetDisplacementShift());
>
> gridTransform->SetDisplacementScale(transformToGrid->GetDisplacementScale());
>
>   vtkSmartPointer< vtkImageReslice > reslice =
> vtkSmartPointer<vtkImageReslice >::New();
>   reslice->SetInput(input);
>   reslice->WrapOn();
>   reslice->SetResliceTransform(gridTransform);
>   reslice->SetInterpolationModeToLinear();
>   reslice->SetOutputSpacing(input->GetSpacing());
>   reslice->SetOutputExtent(0, 255, 0, 255, 0, 2);
>   reslice->Update();
>
> Thanks for your support,
>
> Jorge.
>
>
> David Gobbi wrote:
>>
>> Hi Jorge,
>>
>> For real-time image warping in VTK, you would probably have to go with
>> 256x256 images in order to achieve the 10Hz goal that you are aiming
>> for.  For warping a 512x512x100 volume in real time, I think your only
>> option would be to use the GPU.
>>
>>   David
>>
>>
>> On Thu, Apr 8, 2010 at 4:38 AM, Jorge Ballesteros
>> <jballesteros at motivando.me> wrote:
>>
>>>
>>> Hello all,
>>>
>>> I have been trying to perform a real-time image warping with no success.
>>>  I
>>> have managed to warp a 512x512x100 Image but the computation time it
>>> takes
>>> is very high. Then I extracted a portion of the image, just 3 slices and
>>> warped the image but still don't meet real-time requirements.
>>>
>>> FYI i have been following VTK Examples,
>>> Hybrid/Testing/TestGridWarp3d.tcl,
>>> using the vtkImageReslice class with a vtkGridTransform as the input
>>> transform and a 512x512x3 input image, obtaining a ~10 Hz frame rate.
>>>
>>> Don't know if there is any other way on doing this. It would be really
>>> helpful if you could point me on how to achieve it.
>>>
>>> Thanks,
>>>
>>> Jorge