[CMake] Problems building a MODULE library (OpenFX plugin) with correct cuda object linking

[dev at irieger.net]

Hello,

after hours and hours of searching and trying to get it to work I hope 
someone here can help me as I'm now out of ideas. I'm trying to build an 
OpenFX plugin for DaVinci Resolve (a movie color grading software, 
https://www.blackmagicdesign.com/products) via CMake. Resolve is 
delivered with a developer sample plugin using a standard Makefile (Mac) 
and project files for Apple XCode/VisualStudio. But for a better 
structure I wanted to build a CMake build framework for this plugin.

Notes for better understanding: The CUDA implementation is a proprietary 
Add On to the OpenSource OpenFX interfaces as far as I understand it. 
Stock OpenFX would pass memory references only to host memory address 
space while the Resolve extension passed CUDA/OpenCL addresses to reduce 
memory transfer overhead for better real time processing. OpenFX plugins 
are build as bundles of objects(?) (MODULE library build in CMake).

The basic structure is a plugin that offers processing with CPU, OpenCL 
or CUDA depending on the processing mode the host application is using. 
This plugin is bundled into a single module library which is loaded by 
Resolve. For the plugin a few helper classes are used to handle the 
Plugin metadata (user interface, host communication) while CUDA/OpenCL 
control code is also seperated into two separate files (OpenCLKernel.cpp 
and CudaKernel.cu). Both contain the GPU Kernel itselt and a host 
function which is included in the main Plugin just via extern 
declaration. Sample:
> extern void RunCudaKernel(int p_Width, int p_Height, float* p_Gain, 
> const float* p_Input, float* p_Output);

The Makefile then just compiles all cpp files with the standard $(CXX) 
compiler and the CudaKernel.cu with $(NVCC) which is defined before. 
Then a simple
> $(CXX) -bundle $^ -o $@ -L${CUDAPATH}/lib -lcuda -lcudart 
> -F/Library/Frameworks -framework CUDA -framework OpenCL
does the linking and works fine.


I tried to replicate this structure of linking in CMake and it works as 
long as I keep CUDA deactivated (One of the reasons of building a 
flexible CMake infrastructure is making a few parts configurable via 
CMake/Compiler options). Most stuff is working now, but I have a problem 
integrating the CUDA part.

For OpenCL I just build an object library from the OpenCLKernel.cpp and 
add it as a target object:
> add_library(GainLibOpenCL OBJECT OpenCLKernel.cpp)
...
> add_library(${PLUGIN_NAME} MODULE src/${PLUGIN_NAME}.cpp 
> $<TARGET_OBJECTS:GainLibOpenCL>)

I tried to replicate this for the CUDA code but cuda_add_library doesn't 
seem to support generating object files and a combination of 
cuda_compile + add_library also doesn't allow to generate object files 
so how do I replicate the Makefile result.

Here are two examples how I tried different methods. Did a few more 
tests based on many results of questions on the net but non solved my 
problem.

Variant 1:
>  cuda_add_library(GainLibCUDA CudaKernel.cu)
>  set(LIBS ${LIBS} GainLibCUDA PARENT_SCOPE)
Will compile but result in a crash of Resolve, with STATIC as an option 
for cuda_add_library it builds and Resolve stays open, but processing 
isn't working. And it links to the CUDA static libraries which isn't 
intended as the host cuda lib should be used.

>  cuda_compile(cuda_exec_obj CudaKernel.cu)
>  add_library(GainLibCUDA OBJECT CudaKernel.h ${cuda_exec_obj})
>  set(TARGET_OBJECTS ${TARGET_OBJECTS} $<TARGET_OBJECTS:GainLibCUDA> 
> PARENT_SCOPE)

Results in the error:
> CMake Error at src/CUDA/CMakeLists.txt:21 (add_library):
>   OBJECT library "GainLibCUDA" contains:
> 
>     cuda_compile_1_generated_CudaKernel.cu.o
> 
>   but may contain only sources that compile, header files, and other 
> files
>   that would not affect linking of a normal library.

Also just adding "cuda_compile(cuda_exec_obj CudaKernel.cu)" and then 
${cuda_exec_obj} as a file to the "add_library(${PLUGIN_NAME} MODULE 
..." line results in and error: "Cannot find source file".

So this mail got pretty long but I hope this helps someone to help me 
and then also help others if they reach a similar problem.


Kind regards,
Ingmar