<div dir="ltr"><div>Hi,</div><div>Chao's observations are correct.<br></div><div>- Yes, it's the time per execution. The
backprojection filter was called 43 times with an average of 0.0275691
s, so 1.185 s in total. The 0.025 second difference with the info
displayed on your command line is not completely surprising because the timing done automatically for each filter starts/stops the timing a
bit after/before. Note that we have removed any other timing than RTK_TIME_EACH_FILTER in the current version.<br><div>- <wbr>CudaFDKConeBeamReconstructionF<wbr>illter is a "mini-pipeline" (see ITK doc) of a few filters, as documented <a href="http://www.openrtk.org/Doxygen/classrtk_1_1FDKConeBeamReconstructionFilter.html">here</a>.
The ExtractImageFilter is missing from the drawing but the sum of the 4
(ExtractImagefilter, FDKWeightProjectionFilter, FFTRampImageFilter and
FDKBackprojectionImageFilter) gives 43*(<span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">0.0130324+0.0275</span></span>+0.0389145+0.0275691)=4.6 s. It's different from the 9.6 s you observe and I'm afraid I don't know why. But I did not see FDKWeightProjectionFilter in your list, did you remove some filters from the list? That might explain the few seconds missing if I realize some filters are not there.<br></div>- I'm not sure <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">CUDAWeighting</span></span> is the longest... Cuda computation is asynchronous so the backprojection, which should be the longest, might be finishing in the weighting filter. If you want a more accurate timing, you probably need to force synchronous computation.</div><div>I hope this helps.</div><div>Simon<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 20, 2018 at 1:03 PM, Chao Wu <span dir="ltr"><<a href="mailto:wuchao04@gmail.com" target="_blank">wuchao04@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi, By a quick look, the time reported with RTK_TIME_EACH_FILTER <wbr>seems to be time per execution of each filter. I didn't look into the code so I have no idea whether it is an average time or the time of the last
<span style="font-size:small;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">execution<span> </span></span>. In addition (not shown in your example) if one filter has more than one instances in the pipeline, the report only lists the total number of executions of all instances.<div>Regards, Chao</div></div><br><div class="gmail_quote"><div dir="ltr">Elena Padovani <<a href="mailto:elenapadovani.lk@gmail.com" target="_blank">elenapadovani.lk@gmail.com</a>> 于2018年7月19日周四 上午11:44写道：<br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Simon,<div>Thank you for the fast reply. i changed the RTK_CUDA_PROJECTIONS_SLAB_SIZE but unfortunately nothing has changed. I also compiled it with the FLAG RTK_TIME_EACH_FILTER on and i did not understand why it tells me that <wbr>CudaFDKBackProjectionImageFilt<wbr>er took 0.0275 s, <wbr>CudaFDKConeBeamReconstructionF<wbr>ilter took 9.58 s and CudaFFTRampImageFilter took 0.0389 s while the PrintTiming method tells me that; Prefilter operations took 6.65, Ramp Filter 1.71, Backprojection: 1.21. So my question is where is the remaining time spent ? For instance, is (<span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Backprojection=1.21 - <span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">CudaFDKBackProjectionImageFi<wbr>lter=0.0275) the time needed to copy the memory from CPU to GPU? The same holds for the ramp filter. </span></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Moreover it seems to me that what is taking long is the CUDAWeighting filter so do you think that increasing the number of thread per block which is now { 16, 16 , 2 } could help ? </span></span></div><div><br></div><div>Here is what the applications shows me with the -v option:</div><div><br></div><div><div>Reconstructing and writing... It took 11.8574 s</div><div>FDKConeBeamReconstructionFilte<wbr>r timing:</div><div> Prefilter operations: 6.65107 s</div><div> Ramp filter: 1.71472 s</div><div> Backprojection: 1.21037 s</div><div><br></div><div><br></div><div>******************************<wbr>******************************<wbr>*********************</div><div>Probe Tag Starts Stops Time (s) </div><div>******************************<wbr>******************************<wbr>*********************</div><div>ConstantImageSource 1 1 0.0962241 </div><div>CudaCropImageFilter 43 43 0.00230094 </div><div>CudaFDKBackProjectionImageFilt<wbr>er 43 43 0.0275691 </div><div>CudaFDKConeBeamReconstructionF<wbr>ilter 1 1 9.58291 </div><div>CudaFFTRampImageFilter 43 43 0.0389145 </div><div>ExtractImageFilter 43 43 0.0130324 </div><div>FFTWRealToHalfHermitianForward<wbr>FFTImageFilter 12 12 0.00128049 </div><div>ImageFileReader 686 686 0.0481416 </div><div>ImageFileWriter 1 1 11.8383 </div><div>ImageSeriesReader 686 686 0.0484766 </div><div>ProjectionsReader 1 1 44.7685 </div><div>Self 129 129 0.0506474 </div><div>StreamingImageFilter 2 2 27.713 </div><div>VarianObiRawImageFilter 686 686 0.0135297</div></div><div><br></div><div>At the beginning i was using my own application with my own data i now switched back to the wiki VarianRecontruction test ( with a 512^3 reconstructed volume). </div><div><br></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Thank you again,</span></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Kind Regards</span></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Elena </span></span></div></div><div class="gmail_extra"><br><div class="gmail_quote">2018-07-18 22:00 GMT+02:00 Simon Rit <span dir="ltr"><<a href="mailto:simon.rit@creatis.insa-lyon.fr" target="_blank">simon.rit@creatis.insa-lyon.<wbr>fr</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div>Thanks for sharing your results.</div><div>RTK uses CUFFT for the ramp filtering which does its own blocks/grid management. For backprojection, it's pretty simple, see</div><div><a href="https://github.com/SimonRit/RTK/blob/master/src/rtkCudaFDKBackProjectionImageFilter.cu#L198" target="_blank">https://github.com/SimonRit/<wbr>RTK/blob/master/src/<wbr>rtkCudaFDKBackProjectionImageF<wbr>ilter.cu#L198</a></div><div>mostly hardcoded, independent of the number of CUDA cores and could be optimized. There is one compilation parameter that you can try to change to see if that speeds up the computation, that is the cmake variable RTK_CUDA_PROJECTIONS_SLAB_SIZE which controls how many projections are backprojected simultaneously.</div><div>We currently currently don't propose any way to use multiple GPUs.</div><div>Please keep us posted if you continue to do some tests. In particular, I advise turning on RTK_TIME_EACH_FILTER in cmake so that you get a report with -v option in applications on how much time your program spent in each filter.</div><div>Best regards,</div><div>Simon<br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_-849454065488513647m_-2075478347692395170h5">On Wed, Jul 18, 2018 at 6:48 PM, Elena Padovani <span dir="ltr"><<a href="mailto:elenapadovani.lk@gmail.com" target="_blank">elenapadovani.lk@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-849454065488513647m_-2075478347692395170h5"><div dir="ltr">Hi RTK-users,<div><br></div><div>I compiled RTK with CUDA and tried to setup a benchmark to analyze the performances trend of the GPUs when using the CUDA-FDK reconstruction filter. Precisely, when reconstructing the same volume from the same data-set on NVS510 GTX860M and GTX970M i got results consistent with the number of GPUs cuda cores. Indeed, when setting up this benchmark i was expecting <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">a reduction in the reconstruction time with the increase of cuda cores(at least until the dimension of the reconstructed volume was not the actual bottleneck). </span>However, when testing it on a Tesla P100 i got performances comparable to the GTX860M. Would you expect such a result? </div><div><br></div><div>Unfortunately i am new to CUDA and i was wondering if any of you could help me figuring this out. <br></div><div>How does rtk with CUDA manage the number of blocks/grid dimension ? </div><div>Is the number of blocks/grid dimension depedent on the GPU cuda cores?</div><div>Is there a way to use multiple GPUs?</div><div><br></div><div>The test was carried with the following data:</div><div>- 360 projections</div><div>- reconstructed volume 600x700x800 px</div><div><br></div><div>Thank you in advance </div><div>Kind regards</div><span class="m_-849454065488513647m_-2075478347692395170m_-4559019070186694649HOEnZb"><font color="#888888"><div><br></div><div>Elena </div><div><br></div></font></span></div>
<br></div></div>______________________________<wbr>_________________<br>
Rtk-users mailing list<br>
<a href="mailto:Rtk-users@public.kitware.com" target="_blank">Rtk-users@public.kitware.com</a><br>
<a href="https://public.kitware.com/mailman/listinfo/rtk-users" rel="noreferrer" target="_blank">https://public.kitware.com/<wbr>mailman/listinfo/rtk-users</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>
______________________________<wbr>_________________<br>
Rtk-users mailing list<br>
<a href="mailto:Rtk-users@public.kitware.com" target="_blank">Rtk-users@public.kitware.com</a><br>
<a href="https://public.kitware.com/mailman/listinfo/rtk-users" rel="noreferrer" target="_blank">https://public.kitware.com/<wbr>mailman/listinfo/rtk-users</a><br>
</blockquote></div></div></div>
</blockquote></div><br></div>