[Rtk-users] rtkfdk doesn't work correctly after a recent update

Fri Nov 20 12:36:51 EST 2015

Hi Simon,

That sounds great! Thanks a lot for your hard work.

As you said, now I can reconstruct it successfully with --subsetsize option.

I think the only concern left is the elongated reconstruction time.

I’ve measured it for the same phantom image with different “subsetsize” values as well as w/o FFTW compiling.

I just report it for your information.

1)      w/o FFTW  : 21.4 s

2)      w/ FFTW & --subsetsize 1: 89 s

3)      w/ FFTW & --subsetsize 2: 77 s

4)      w/ FFTW & --subsetsize 4: 65 s

5)      w/ FFTW & --subsetsize 6: 51 s

6)      w/ FFTW & --subsetsize 7: (Error-graphic card seemed to be freezed)

7)      w/ FFTW & --subsetsize 8: (Error-“itk::ERROR: CUFFT ERROR #2 in rtkCudaFFTConvolutionImageFilter.cu”)

Of course, the difference was coming from the Ramp filter.

Therefore, I’m going to use [--subsetsize 6] for my system and will update CUDA version soon!

Thanks.

Yang

From: simon.rit at gmail.com [mailto:simon.rit at gmail.com] On Behalf Of Simon Rit
Sent: Friday, November 20, 2015 11:28 AM
To: Yang-Kyun Park <theday79 at gmail.com>
Cc: rtk-users at openrtk.org
Subject: Re: [Rtk-users] rtkfdk doesn't work correctly after a recent update

Hi Yang,
Thanks a lot for sharing the data. I have good news. I could reproduce the bug and I have solutions. First, about your dataset: it's a huge dataset and the hannY requires zero padding in both x and y. Because of this, cufft has to take the 2D FFT of projection subsets that are 4095x2048x16. Quite big!

When I ran it on my cuda 7.0, I immediately got:

itk::ExceptionObject (0x8b9cb30)
Location: "void CUDA_fft_convolution(const int3&, const int2&, float*, cufftComplex*)" 
File: /home/srit/src/rtk/rtk/code/rtkCudaFFTConvolutionImageFilter.cu
Line: 97
Description: itk::ERROR: CUFFT ERROR #2

i.e., a GPU alloc issue, even if I have the largest possible memory on a nvidia desktop computer. Then, I ran it with cuda 6.5 and I got no error message but a result similar to yours.

Conclusions:

- if you add the option --subsetsize 1 to your rtkfdk command line, that should reconstruct correctly

- if you upgrade cuda, you should get an understandable error message,

- I don't think I'm going to look for a solution because the pb is probably in cufft but if someone is willing to, be my guest!

Simon

On Mon, Nov 16, 2015 at 4:47 PM, Yang-Kyun Park <theday79 at gmail.com <mailto:theday79 at gmail.com> > wrote:

Hi Simon,

For better understanding, I'm attaching the problematic case including projection files, geometry file(xml) and reconstructed images.(please find the GDrive links. The file size is ~800 MB)

This is phantom data and there is no patient information in it.

Regarding the fftw library, I'm using [fftw-3.3.4-dll64].

By the way, I found another clue. When I've tried it with a head phantom image (full-fan, ~ 190 projections), there was no problem. Is there any possibility that DDF and parker short filter play some roles on this issue?

Feel free to let me know if further data is needed.

Thanks!

Yang

 <https://drive.google.com/file/d/0BxuP3mON9V-qbVdQd0luc1lIb2M/view?usp=drive_web>  FFTW_issue.zip

On Mon, Nov 16, 2015 at 10:07 AM, Simon Rit <simon.rit at creatis.insa-lyon.fr <mailto:simon.rit at creatis.insa-lyon.fr> > wrote:

Hi,

Thanks. It's really weird... When you run the tests with FFTW (compiling the "test" target), do they fail?
Can you provide your geometry file? It might be specific to this geometry.

Thanks again for looking into this issue, we'll find what's the problem.

Simon

On Mon, Nov 16, 2015 at 3:54 PM, Yang K Park <theday79 at gmail.com <mailto:theday79 at gmail.com> > wrote:

Hi Simon,

Sorry for the late response.

As you suggested, I’ve tried rtkramp+CUDA with only one projection and it was successfully done for both w/ FFTW and w/o FFTW.

(Two results were exactly same.)

Regarding  the CUDA version, I’m using v6.5.

Thanks a lot for the help!

Yang

From: simon.rit at gmail.com <mailto:simon.rit at gmail.com>  [mailto:simon.rit at gmail.com <mailto:simon.rit at gmail.com> ] On Behalf Of Simon Rit
Sent: Thursday, November 12, 2015 5:59 PM

To: Yang K Park <theday79 at gmail.com <mailto:theday79 at gmail.com> >
Cc: rtk-users at openrtk.org <mailto:rtk-users at openrtk.org> 
Subject: Re: [Rtk-users] rtkfdk doesn't work correctly after a recent update

Hi,

Ok. Can you try rtkramp with only one projection? Then you shouldn't encounter this regions issue (that should be solved one day but that's a different story). 

Can you also indicate which version of CUDA you're using?

FYI, we use ITK for the FFT of the kernel of the ramp filter and cufft for the FFT of the projections. So if they are not in the same format, you're in trouble and that might be the issue here... The weird part is that I didn't reproduce this problem on linux (with and without FFTW) but it might be a CUDA version difference. I'll give it another try tomorrow when you provide the CUDA version.

Thanks again for reporting the issue,
Simon

On Thu, Nov 12, 2015 at 11:39 PM, Yang K Park <theday79 at gmail.com <mailto:theday79 at gmail.com> > wrote:

Hi Simon,

Thanks for the advice. The current RTK was compiled with FFTW to use the phase extraction feature from Amsterdam shroud.

Regarding your suggestions,

1)      rtkramp w/ or w/o CUDA:

(command line: “ rtkramp --path projPath --regexp .*.his --hardware cuda --output Ramp_wCUDA.mha” )

Then, I have encountered following error.

“CudaFFTRampImageFilter<..> CudaFFTConvolutionImageFilter assumes that input requested and buffered regions are equal.”

Without CUDA, it was successfully done.

2)      I’ve re-compiled ITK w/o FFTW and built RTK again. rtkfdk worked fine.

-          I’ve also repeated “rtkramp+CUDA” with this rtk version and the error was same as above.

In conclusion, my FFTW seems to affect CUDA-based fdk reconstruction only.

But I do need FFTW for the phase extraction. What should I do?

Thanks.

Yang

-----Original Message-----
From: simon.rit at gmail.com <mailto:simon.rit at gmail.com>  [mailto:simon.rit at gmail.com <mailto:simon.rit at gmail.com> ] On Behalf Of Simon Rit
Sent: Wednesday, November 11, 2015 4:16 PM
To: Yang K Park <theday79 at gmail.com <mailto:theday79 at gmail.com> >
Cc: rtk-users at openrtk.org <mailto:rtk-users at openrtk.org> 
Subject: Re: [Rtk-users] rtkfdk doesn't work correctly after a recent update

Hi,

I didn't manage to reproduce the two problems on linux with the same ITK version.

For point 1, did you copy itkMacro.h by hand? Because I don't see how replacing <itkMacro.h> by "itkMacro.h" can help if itkMacro.h is not in the same folder as rtkCudaUtilities.hcu For point 2, no clue. Do you use FFTW? It looks like a ramp filter issue. Can you try rtkramp --path projPath --regexp .*.his, with and without --hardware cuda? Maybe you can provide your geometry file in case it's a specific geometric issue.

Simon

On Wed, Nov 11, 2015 at 7:50 PM, Yang K Park < <mailto:theday79 at gmail.com> theday79 at gmail.com> wrote:

> Hi all,

> 

> 

> 

> I’ve updated rtk with the up-to-date revision and have encountered a 

> couple of problems.

> 

> 

> 

> 1.       Compile error: RTK couldn’t find itkMacro.h while it compiled

> rtkCudaUtilities.hcu. It was solved by a minor fetching:

> 

> #include <itkMacro.h>   à #include "itkMacro.h" in rtkCudaUtilities.hcu

> 

> 

> 

> 2.       Output of rtkfdk with CUDA was not the same as before. It was very

> blurry as seen in the attached image.

> 

> a.       With “FDK with CPU” or “SART with CUDA”, there was no problem.

> 

> 

> 

> 

> 

> I’m using VS2013 with win7, itk4.8.1.

> 

> 

> 

> The command line I used was :

> 

> rtkfdk --geometry 1111.xml --path projPath --verbos --regexp .*.his 

> --output output.mha --spacing 1,1,1 --dimension 400,200,400 --hardware 

> cuda --pad 1 --hann 5 --hannY 5

> 

> 

> 

> Does anyone have any idea about this?

> 

> 

> 

> Thanks.

> 

> 

> 

> Yang

> 

> 

> 

> 

> _______________________________________________

> Rtk-users mailing list

>  <mailto:Rtk-users at public.kitware.com> Rtk-users at public.kitware.com

>  <http://public.kitware.com/mailman/listinfo/rtk-users> http://public.kitware.com/mailman/listinfo/rtk-users

> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/rtk-users/attachments/20151120/0d98ace1/attachment-0010.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 5752 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/rtk-users/attachments/20151120/0d98ace1/attachment-0010.png>