[Rtk-users] How to run the reconstruction on a specific GPU?

[ghostcz]

Dear Cyril,

Thank you for the follow-up.
That is indeed a faster way to realize this function. I wrote a launcher 
instead, where I call cuda_set_device to select the device.
No, I don’t think I know CUDA good enough to manage the memory transfer 
between the devices.

Best regards,
Louie

P.S. is there any training or workshop focusing on rtk application and 
development available?




-----Original Message----- 
From: Cyril Mory
Sent: Wednesday, September 17, 2014 11:59 AM
To: Simon Rit ; ghostcz
Cc: rtk-users at openrtk.org
Subject: Re: [Rtk-users] How to run the reconstruction on a specific GPU?

Hi Louie,

We have been investigating your question bit. A first possible solution,
which may be sufficient for your needs, could be to set the
CUDA_VISIBLE_DEVICES environment variable. You'll find details here :
http://www.acceleware.com/blog/cudavisibledevices-masking-gpus

Here are two scripts I used to run tests on our two-GPUs server:

######### First script #########
echo "Running two instances of rtkfdk without specifying a GPU"
CUDA_VISIBLE_DEVICES="0,1"
time rtkfdk -p . -r .*.his -o fdkBothGPUs_0.mha --hardware cuda -g
geom.xml --dimension 512 &
time rtkfdk -p . -r .*.his -o fdkBothGPUs_1.mha --hardware cuda -g
geom.xml --dimension 512 &

######### Second script #########
echo "Running two instances of rtkfdk, specifying which GPU to use for each"
echo 'CUDA_VISIBLE_DEVICES="0"; time rtkfdk -p . -r .*.his -o
fdkGPU_0.mha --hardware cuda -g geom.xml --dimension 512 &' | bash
echo 'CUDA_VISIBLE_DEVICES="1"; time rtkfdk -p . -r .*.his -o
fdkGPU_1.mha --hardware cuda -g geom.xml --dimension 512 &' | bash

######### End #############

On our server, which has two identical GPUs, these two scripts took the
same time. It seems that, in the first script, rtkfdk somehow managed to
detect that the first GPU was in use and ran on the second one. But I
can't guarantee under which conditions this happens, and surely it's
safer to use the second script.

This is the cheapest solution. If you need more than that, and you are
ready to dive into the cuda filters' code as well as into the automatic
transfers between CPU and GPU memory, do not hesitate to propose
modifications of the RTK code.

Regards,
Cyril

On 09/15/2014 02:45 PM, Simon Rit wrote:
> Hi,
> Good question, I did not hear anybody working on multiple GPUs so far.
> It currently uses the first Cuda compatible device. I think you'll
> have to look up in the Cuda manual to see how to switch to another one
> but there is nothing available in the RTK lib yet. Don't hesitate to
> suggest some code changes to add this feature.
> Simon
>
> On Mon, Sep 15, 2014 at 2:09 PM, ghostcz <ghostcz at hotmail.com> wrote:
>> Dear Rtk users,
>>
>> How can I select one (or maybe more?) specific GPU to run my 
>> reconstruction?
>> It seems if I start a second instance of the reconstruction program, it 
>> will
>> run on the same GPU.
>>
>> Best regards,
>> Louie
>>
>> _______________________________________________
>> Rtk-users mailing list
>> Rtk-users at public.kitware.com
>> http://public.kitware.com/mailman/listinfo/rtk-users
>>
> _______________________________________________
> Rtk-users mailing list
> Rtk-users at public.kitware.com
> http://public.kitware.com/mailman/listinfo/rtk-users
>

-- 
--
Cyril Mory, Post-doc
CREATIS
Leon Berard cancer treatment center
28 rue Laënnec
69373 Lyon cedex 08 FRANCE

Mobile: +33 6 69 46 73 79