[CMake] FindCUDA ignores project dependencies when separable compilation is on

[Irwin Zaid]

Okay, an update on this.

1) This is fixed now, thank you. We just needed to add a custom target, as you said.

2) This is trickier and, unfortunately, still not working. We are already adding -fPIC to CMAKE_CXX_FLAGS, should that not be enough? I also tried adding it to both CMAKE_CXX_FLAGS_DEBUG and CMAKE_CXX_FLAGS_RELEASE, with no effect.

Looking into FindCUDA.CMake at CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS, I find code very similar to what you are describing. Are you saying that in addition to what is below, we need to add what you proposed? This is what I see.

--

foreach(config ${CUDA_configuration_types})
string(TOUPPER ${config} config_upper)
# Add config specific flags
foreach(f ${CUDA_NVCC_FLAGS_${config_upper}})
list(APPEND config_specific_flags $<$<CONFIG:${config}>:${f}>)
endforeach()
set(important_host_flags)
_cuda_get_important_host_flags(important_host_flags ${CMAKE_${CUDA_C_OR_CXX}_FLAGS_${config_upper}})
foreach(f ${important_host_flags})
list(APPEND flags $<$<CONFIG:${config}>:-Xcompiler> $<$<CONFIG:${config}>:${f}>)
endforeach()
endforeach()


James Bigler wrote:
> 2. It looks as though CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS is only 
> looking at the configuration specific flags. You can add the flag 
> specifically to all your configs (e.g. CMAKE_CXX_FLAGS_DEBUG) or you 
> could try adding these lines of code in your FindCUDA.cmake file 
> somewhere in CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS after set(flags). 
> It looks as though this was overlooked (sorry).
>
> set(important_host_flags)
> _cuda_get_important_host_flags(important_host_flags 
> ${CMAKE_${CUDA_C_OR_CXX}_FLAGS})
> foreach(f ${important_host_flags})
> list(APPEND flags -Xcompiler ${f})
> endforeach()
>
>
>
> On Mon, Jan 5, 2015 at 3:43 PM, Irwin Zaid 
> <irwin.zaid at physics.ox.ac.uk <mailto:irwin.zaid at physics.ox.ac.uk>> wrote:
>
>     Alright, this is a lot of progress!
>
>     1) We are using Makefiles. I agree with you about the dependency
>     graph, so I'll try and sort that out. I'll let you know what the
>     result is.
>
>     2) I just checked and, indeed, the *_intermediate_link.o file is
>     not being passed -fPIC. Is this our problem? What is the correct fix?
>
>     Irwin
>
>     James Bigler wrote:
>
>
>
>         On Mon, Jan 5, 2015 at 1:57 PM, Irwin Zaid
>         <irwin.zaid at physics.ox.ac.uk <mailto:irwin.zaid at physics.ox.ac.uk>
>         <mailto:irwin.zaid at physics.ox.__ac.uk
>         <mailto:irwin.zaid at physics.ox.ac.uk>>> wrote:
>
>         Hi James,
>
>         Thanks for the quick reply! As I mentioned, we've hit two issues.
>         The first is the project dependencies one, which I'll try and
>         describe more a bit below. I'm not a CMake expert, so please bear
>         with me.
>
>         The second is what I've put under 2).
>
>         The only CMake build dependency changes when doing separable
>         compilation
>         are found in CUDA_LINK_SEPARABLE_____COMPILATION_OBJECTS.
>         Basically what
>         this does is create a new rule to build an intermediate link
>         file. For
>         everything but some versions of MSVC generators it adds a custom
>         command
>         to generate this intermediate link file. The other case adds a
>         custom
>         command that runs as a PRE_LINK command to generate the object
>         file (the
>         reasons for this is a bug in VS custom command dependency
>         scanning), but
>         this should happen during link phase and not compile phase.
>
>         Nothing in here should change what happens before the target is
>         built,
>         though.
>
>
>         1) Okay, I understand that, but I do think we saw a different
>         behaviour when we switched to separable compilation. Let me
>         describe
>         what we are doing.
>
>         We generate part of our library from a simple program (call the
>         simple program 'gen', which generates a source file 'gen.hpp')
>         that
>         we want to execute before compiling our library (call our library
>         'main'). We set this up with the following:
>
>         - add_executable(...) is called for 'gen'
>         - add_custom_command(...) sets up a command that executes 'gen'
>         - set_source_files_properties(..____.) is called to set
>         'gen.hpp' as
>         having the PROPERTY of GENERATED
>         - add_dependencies(main gen) is called to establish 'main' depends
>         on 'gen'
>
>         So far, this has only failed for CUDA with separable compilation.
>         (It has worked for all of our other configurations. including CUDA
>         without separable compilation.)
>
>         Have we done something wrong? Is there some additional information
>         we can look at to figure out what's going on?
>
>
>         What kind of generator are you using (e.g. makefile)?
>
>         Here's what I'm thinking might be the problem, though I'm not
>         sure why
>         it would have worked without CUDA_SEPARABLE_COMPILATION.
>
>         There's a dependency between gen and gen.hpp (encoded in the
>         call to
>         add_custom_command(OUTPUTS gen.hpp COMMAND gen)).
>         There's a dependency between main and gen (can't start
>         building main
>         until gen is built).
>         There's a dependency between gen.hpp and main (gen.hpp is an input
>         source to main, so it needs to be built as part of main).
>
>         What I don't see is a dependency between gen.hpp and all the cuda
>         sources that might use it as input. So from a dependency graph
>         standpoint a makefile (if one is being used in this case) is
>         entirely
>         free to start compiling the CUDA code once the gen target has been
>         satisfied.
>
>         What you need is another target that builds the gen.hpp file
>         which can
>         be forced to run before main starts to build. There are more
>         than one
>         way to do this, and I'm not sure what the best option is, but
>         you might
>         try this:
>
>         add_custom_command(OUTPUTS gen.hpp ....)
>         add_custom_target(make_gen_hpp DEPENDS gen.hpp)
>         add_dependency(main make_gen_hpp)
>
>         2) A second problem we've run into is an error when trying to
>         link a
>         CUDA shared library with separable compilation. This is
>         specifically
>         a Linux problem, on Mac it is fine. A static library is also fine,
>         working for both Linux and Mac.
>
>         The particular error is "relocation R_X86_64_32S against
>         `__nv_module_id' can not be used when making a shared object;
>         recompile with -fPIC". However, we are already compiling with
>         -fPIC.
>         I can confirm that -fPIC appears to be passed to both the host
>         compiler for non-CUDA source and via -Xcompiler -fPIC for CUDA
>         source.
>
>         This error occurs when trying to link all the different object
>         files
>         together of our library.
>
>         Do you have any idea of what this could be?
>
>
>         I'm not sure which object file wasn't compiled with -fPIC, but
>         I would
>         suspect it might be the intermediate link object file. FindCUDA is
>         supposed to pass this flag along (see
>         function(_cuda_get_important___host_flags)), but you might
>         want to verify
>         this with a 'make VERBOSE=1' and look for
>         <target_name>_intermediate___link.o (substitute your target
>         name in or
>         leave it out of the search string).
>
>         James
>
>