[Rtk-users] How to run the reconstruction on a specific GPU?

[Cyril Mory]

Hi Louie,

Good that you managed to solve your problem.

Your question on training made us think a bit. Here are a few answers:
- At the moment, there is only a wiki, the documentation and this 
mailing list
- We're only a handful of developers working on RTK, so it seems too 
early, and too much logistics, to organize real workshops or training 
sessions in the near future
- That being said, we would like to leave the door open. Surely it is 
important, if we want more people to use RTK and contribute to it, that 
new developers can learn easily and efficiently. I'll start a specific 
thread on the mailing list to gather the possible requests similar to yours
- RTK uses ITK to do most of the work, therefore programming in RTK 
essentially requires knowing ITK and knowing some tomography. ITK 
training sessions are organized often and in many countries

I hope it helps,
Cyril

On 09/17/2014 05:58 PM, ghostcz wrote:
> Dear Cyril,
>
> Thank you for the follow-up.
> That is indeed a faster way to realize this function. I wrote a 
> launcher instead, where I call cuda_set_device to select the device.
> No, I don’t think I know CUDA good enough to manage the memory 
> transfer between the devices.
>
> Best regards,
> Louie
>
> P.S. is there any training or workshop focusing on rtk application and 
> development available?
>
>
>
>
> -----Original Message----- From: Cyril Mory
> Sent: Wednesday, September 17, 2014 11:59 AM
> To: Simon Rit ; ghostcz
> Cc: rtk-users at openrtk.org
> Subject: Re: [Rtk-users] How to run the reconstruction on a specific GPU?
>
> Hi Louie,
>
> We have been investigating your question bit. A first possible solution,
> which may be sufficient for your needs, could be to set the
> CUDA_VISIBLE_DEVICES environment variable. You'll find details here :
> http://www.acceleware.com/blog/cudavisibledevices-masking-gpus
>
> Here are two scripts I used to run tests on our two-GPUs server:
>
> ######### First script #########
> echo "Running two instances of rtkfdk without specifying a GPU"
> CUDA_VISIBLE_DEVICES="0,1"
> time rtkfdk -p . -r .*.his -o fdkBothGPUs_0.mha --hardware cuda -g
> geom.xml --dimension 512 &
> time rtkfdk -p . -r .*.his -o fdkBothGPUs_1.mha --hardware cuda -g
> geom.xml --dimension 512 &
>
> ######### Second script #########
> echo "Running two instances of rtkfdk, specifying which GPU to use for 
> each"
> echo 'CUDA_VISIBLE_DEVICES="0"; time rtkfdk -p . -r .*.his -o
> fdkGPU_0.mha --hardware cuda -g geom.xml --dimension 512 &' | bash
> echo 'CUDA_VISIBLE_DEVICES="1"; time rtkfdk -p . -r .*.his -o
> fdkGPU_1.mha --hardware cuda -g geom.xml --dimension 512 &' | bash
>
> ######### End #############
>
> On our server, which has two identical GPUs, these two scripts took the
> same time. It seems that, in the first script, rtkfdk somehow managed to
> detect that the first GPU was in use and ran on the second one. But I
> can't guarantee under which conditions this happens, and surely it's
> safer to use the second script.
>
> This is the cheapest solution. If you need more than that, and you are
> ready to dive into the cuda filters' code as well as into the automatic
> transfers between CPU and GPU memory, do not hesitate to propose
> modifications of the RTK code.
>
> Regards,
> Cyril
>
> On 09/15/2014 02:45 PM, Simon Rit wrote:
>> Hi,
>> Good question, I did not hear anybody working on multiple GPUs so far.
>> It currently uses the first Cuda compatible device. I think you'll
>> have to look up in the Cuda manual to see how to switch to another one
>> but there is nothing available in the RTK lib yet. Don't hesitate to
>> suggest some code changes to add this feature.
>> Simon
>>
>> On Mon, Sep 15, 2014 at 2:09 PM, ghostcz <ghostcz at hotmail.com> wrote:
>>> Dear Rtk users,
>>>
>>> How can I select one (or maybe more?) specific GPU to run my 
>>> reconstruction?
>>> It seems if I start a second instance of the reconstruction program, 
>>> it will
>>> run on the same GPU.
>>>
>>> Best regards,
>>> Louie
>>>
>>> _______________________________________________
>>> Rtk-users mailing list
>>> Rtk-users at public.kitware.com
>>> http://public.kitware.com/mailman/listinfo/rtk-users
>>>
>> _______________________________________________
>> Rtk-users mailing list
>> Rtk-users at public.kitware.com
>> http://public.kitware.com/mailman/listinfo/rtk-users
>>
>

-- 
--
Cyril Mory, Post-doc
CREATIS
Leon Berard cancer treatment center
28 rue Laënnec
69373 Lyon cedex 08 FRANCE

Mobile: +33 6 69 46 73 79