[CMake] FindCUDA ignores project dependencies when separable compilation is on
James Bigler
jamesbigler at gmail.com
Mon Jan 5 19:02:50 EST 2015
2. It looks as though CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS is only
looking at the configuration specific flags. You can add the flag
specifically to all your configs (e.g. CMAKE_CXX_FLAGS_DEBUG) or you could
try adding these lines of code in your FindCUDA.cmake file somewhere
in CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS after set(flags). It looks as
though this was overlooked (sorry).
set(important_host_flags)
_cuda_get_important_host_flags(important_host_flags
${CMAKE_${CUDA_C_OR_CXX}_FLAGS})
foreach(f ${important_host_flags})
list(APPEND flags -Xcompiler ${f})
endforeach()
On Mon, Jan 5, 2015 at 3:43 PM, Irwin Zaid <irwin.zaid at physics.ox.ac.uk>
wrote:
> Alright, this is a lot of progress!
>
> 1) We are using Makefiles. I agree with you about the dependency graph, so
> I'll try and sort that out. I'll let you know what the result is.
>
> 2) I just checked and, indeed, the *_intermediate_link.o file is not being
> passed -fPIC. Is this our problem? What is the correct fix?
>
> Irwin
>
> James Bigler wrote:
>
>>
>>
>> On Mon, Jan 5, 2015 at 1:57 PM, Irwin Zaid <irwin.zaid at physics.ox.ac.uk
>> <mailto:irwin.zaid at physics.ox.ac.uk>> wrote:
>>
>> Hi James,
>>
>> Thanks for the quick reply! As I mentioned, we've hit two issues.
>> The first is the project dependencies one, which I'll try and
>> describe more a bit below. I'm not a CMake expert, so please bear
>> with me.
>>
>> The second is what I've put under 2).
>>
>> The only CMake build dependency changes when doing separable
>> compilation
>> are found in CUDA_LINK_SEPARABLE___COMPILATION_OBJECTS.
>> Basically what
>> this does is create a new rule to build an intermediate link
>> file. For
>> everything but some versions of MSVC generators it adds a custom
>> command
>> to generate this intermediate link file. The other case adds a
>> custom
>> command that runs as a PRE_LINK command to generate the object
>> file (the
>> reasons for this is a bug in VS custom command dependency
>> scanning), but
>> this should happen during link phase and not compile phase.
>>
>> Nothing in here should change what happens before the target is
>> built,
>> though.
>>
>>
>> 1) Okay, I understand that, but I do think we saw a different
>> behaviour when we switched to separable compilation. Let me describe
>> what we are doing.
>>
>> We generate part of our library from a simple program (call the
>> simple program 'gen', which generates a source file 'gen.hpp') that
>> we want to execute before compiling our library (call our library
>> 'main'). We set this up with the following:
>>
>> - add_executable(...) is called for 'gen'
>> - add_custom_command(...) sets up a command that executes 'gen'
>> - set_source_files_properties(..__.) is called to set 'gen.hpp' as
>> having the PROPERTY of GENERATED
>> - add_dependencies(main gen) is called to establish 'main' depends
>> on 'gen'
>>
>> So far, this has only failed for CUDA with separable compilation.
>> (It has worked for all of our other configurations. including CUDA
>> without separable compilation.)
>>
>> Have we done something wrong? Is there some additional information
>> we can look at to figure out what's going on?
>>
>>
>> What kind of generator are you using (e.g. makefile)?
>>
>> Here's what I'm thinking might be the problem, though I'm not sure why
>> it would have worked without CUDA_SEPARABLE_COMPILATION.
>>
>> There's a dependency between gen and gen.hpp (encoded in the call to
>> add_custom_command(OUTPUTS gen.hpp COMMAND gen)).
>> There's a dependency between main and gen (can't start building main
>> until gen is built).
>> There's a dependency between gen.hpp and main (gen.hpp is an input
>> source to main, so it needs to be built as part of main).
>>
>> What I don't see is a dependency between gen.hpp and all the cuda
>> sources that might use it as input. So from a dependency graph
>> standpoint a makefile (if one is being used in this case) is entirely
>> free to start compiling the CUDA code once the gen target has been
>> satisfied.
>>
>> What you need is another target that builds the gen.hpp file which can
>> be forced to run before main starts to build. There are more than one
>> way to do this, and I'm not sure what the best option is, but you might
>> try this:
>>
>> add_custom_command(OUTPUTS gen.hpp ....)
>> add_custom_target(make_gen_hpp DEPENDS gen.hpp)
>> add_dependency(main make_gen_hpp)
>>
>> 2) A second problem we've run into is an error when trying to link a
>> CUDA shared library with separable compilation. This is specifically
>> a Linux problem, on Mac it is fine. A static library is also fine,
>> working for both Linux and Mac.
>>
>> The particular error is "relocation R_X86_64_32S against
>> `__nv_module_id' can not be used when making a shared object;
>> recompile with -fPIC". However, we are already compiling with -fPIC.
>> I can confirm that -fPIC appears to be passed to both the host
>> compiler for non-CUDA source and via -Xcompiler -fPIC for CUDA source.
>>
>> This error occurs when trying to link all the different object files
>> together of our library.
>>
>> Do you have any idea of what this could be?
>>
>>
>> I'm not sure which object file wasn't compiled with -fPIC, but I would
>> suspect it might be the intermediate link object file. FindCUDA is
>> supposed to pass this flag along (see
>> function(_cuda_get_important_host_flags)), but you might want to verify
>> this with a 'make VERBOSE=1' and look for
>> <target_name>_intermediate_link.o (substitute your target name in or
>> leave it out of the search string).
>>
>> James
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20150105/5135cf70/attachment.html>
More information about the CMake
mailing list