View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0011887CMakeCMakepublic2011-02-22 11:322016-06-10 14:31
ReporterPeter Colberg 
Assigned ToKitware Robot 
PrioritynormalSeverityfeatureReproducibilityN/A
StatusclosedResolutionmoved 
PlatformOSOS Version
Product VersionCMake-2-8 
Target VersionFixed in Version 
Summary0011887: Add native CUDA compiler support
DescriptionHi,

CMake includes a FindCUDA module since version 2.8, which adds support for compilation of CUDA source files by means of custom commands such as CUDA_ADD_EXECUTABLE and CUDA_ADD_LIBRARY.

It would be nice to support the CUDA language (although not standardized compared to C or C++) as a first class citizen in CMake. For this purpose, I have been maintaining since a while a patch to CMake which adds native CUDA support. This allows users to compile CUDA source files by simply adding enable_language(CUDA) to CMakeLists.txt. CUDA source files may be included in standard CMake targets, e.g. add_executable and add_library. Besides .o targets, CMake generates .s targets that compile a source to CUDA PTX (intermediate virtual machine code). The patch also includes a _minimal_ FindCUDA.cmake, which searches for the CUDA driver (cuda) and CUDA runtime (cudart) libraries.

Do you see a chance to merge the two efforts in upstream CMake?

(The patch's repository is at http://git.colberg.org/cmake-cuda.git/. [^])

Cheers,
Peter
TagsNo tags attached.
Attached Filespatch file icon cmake-cuda-2.8.3.patch [^] (105,885 bytes) 2011-02-22 11:32 [Show Content]
patch file icon cmake-cuda-2.8.4.patch [^] (105,628 bytes) 2011-03-18 18:23 [Show Content]
patch file icon CMake-CUDA-2.8.4-0-g7f92dd3.patch [^] (105,723 bytes) 2011-03-22 12:41 [Show Content]
patch file icon cmake-cuda-2.8.5-0-gda84f60.patch [^] (105,761 bytes) 2011-07-28 22:35 [Show Content]
patch file icon cmake-cuda-2.8.7-0-ga9ce0fa.patch [^] (108,504 bytes) 2012-01-11 11:34 [Show Content]
patch file icon cmake-cuda-2.8.8-2-ga06ff77.patch [^] (110,157 bytes) 2012-06-20 15:56 [Show Content]

 Relationships

  Notes
(0025532)
Bill Hoffman (manager)
2011-02-22 11:44

 How does this patch interact with VS IDE and Xcode?
(0025533)
Peter Colberg (reporter)
2011-02-22 12:01

 So far the native CUDA patch only supports Linux (see Modules/Platform/Linux-NVCC-CUDA.cmake), as I do not have access to Windows or Mac OS X platforms. I could try to write compiler modules for Windows and Mac OS X, but I lack experience with both platforms and would not able to test on these.

Does anyone of the CMake developers use CUDA?
(0025534)
James Bigler (developer)
2011-02-22 12:17

 Hi there.

The biggest reason that FindCUDA is so complicated is to allow for source level dependencies. If I change a header file included by my .cu file, I want that file to build when I run make. This can be done manually, but maintaining such dependencies manually is often buggy.

I certainly wouldn't mind a more "native" feel for CUDA source files, but whenever I go down that path I get hung up on this.

Also, please keep in mind that the run_nvcc.cmake script that is used to call nvcc during build has been used to work around various bugs in NVCC in the past.

In addition, when you change the flags that nvcc or your host code will use, the build targets will also recompile. This is better than you get with VS+CMake, because VS doesn't recognize the external change to the flags unlike a makefile generator.

Also one of the nice features of FindCUDA is the ability to switch the location of of the CUDA toolkit and have it redo all the variables. Does this support that?
(0025535)
Peter Colberg (reporter)
2011-02-22 12:28

 The dependency scanning of CUDA header files was precisely the reason why I started this patch. IIRC, CMake did not support hooking custom commands into the dependency scanner back in 2008, and the (external) FindCUDA module didn't either.
I have not tested the dependency scanning of the current CMake 2.8 FindCUDA module. But for my patch I know that I never had any trouble with updated CUDA headers triggering a rebuild, it works just as well as with C or CXX headers.

Yes, switching the location of the CUDA toolkit with the "native" patch is straight-forward, e.g. using CMAKE_PREFIX_PATH. To test my code with different CUDA versions, I simply add the appropriate CUDA toolkit path to CMAKE_PREFIX_PATH and create a new build tree.
(0025536)
Peter Colberg (reporter)
2011-02-22 12:38

 > I certainly wouldn't mind a more "native" feel for CUDA source files, but whenever I go down that path I get hung up on this.

I am curious about what kept you from going down this path. I have been using the NVCC compiler without a wrapper script since version 1.1, and so far did not have any issues with the execution of nvcc itself. Is this an issue with NVCC on Windows or Mac OS X?

Peter
(0025537)
James Bigler (developer)
2011-02-22 12:48

 Adding stuff to environment variables and reconfiguring is really annoying in Windows (though it works great in Linux ;) I would prefer it to work the way it does now.

Also, the automatic dependency scanning only works in Makefile systems as per the CMake documentation. I'm curious why the header file dependency didn't work for FindCUDA when you tried it last. This has always been a version 1 feature.
(0025538)
Peter Colberg (reporter)
2011-02-22 13:00

 The dependency scanning in FindCUDA (version from the beginning of 2008) did in fact work, but it sometimes(?) required a manual update of the build tree using cmake before invoking make. Anyway, it was enough motivation to dig into the CMake engine and see how it had been implemented for C and CXX.
(0025539)
Peter Colberg (reporter)
2011-02-22 13:11

 CMAKE_PREFIX_PATH may be initialized using both an environment variable and a CMake cache variable. Does that make switching the toolkit (without using any additional scripts) more comfortable under Windows?
(0025540)
James Bigler (developer)
2011-02-22 13:12

 That particular issue is something I've been trying to track down for a while, but it's extremely intermittent making it difficult to reproduce and fix. I've been experimenting with a fix that might make it better, but I haven't checked it in yet. I'm still waiting to see if the issue reproduces (which it only does in Makefile builds).

That being said, CMake's internal dependency scanning doesn't cover non-Makefile build systems so it was a no go from the start. I wanted the dependency scanning to also work in VS and other IDEs.

This dependency issue is the primary reason why I never implemented CUDA support as a language feature.
(0025541)
Peter Colberg (reporter)
2011-02-22 13:13

 Minor correction: A CMake variable, not a cached CMake variable.
(0025542)
James Bigler (developer)
2011-02-22 13:14

 That's a start, but tt's a lot more straight forward to ask users to set the CUDA_TOOLKIT_ROOT_DIR than to modify CMAKE_PREFIX_PATH. Also, would that allow for "switching" after configure? CUDA puts a single toolkit into the PATH, but it may not be the one you want.
(0025543)
Peter Colberg (reporter)
2011-02-22 13:14

 This may not be the right place to ask, but how is dependency scanning under Windows implemented for C or C++ sources?
(0025544)
James Bigler (developer)
2011-02-22 13:20
edited on: 2011-02-22 13:20

 As far as I can tell, VS just does it internally for cl compiled code. I'm not sure what nmake does.
(0025545)
Peter Colberg (reporter)
2011-02-22 13:57

 > That's a start, but tt's a lot more straight forward to ask users to set the CUDA_TOOLKIT_ROOT_DIR than to modify CMAKE_PREFIX_PATH.

As a user of CMake, I am by no means able to say if CMAKE_PREFIX_PATH is the preferred way to set paths to executables (e.g. compilers), libraries and include headers, but from my limited GNU/Linux point of view, this is the most comfortable one. There is no need to remember special variables for each package, just add the paths to custom-installed libraries (or CUDA toolkit in this case) to CMAKE_PREFIX_PATH and one is done.

With regard to the Unix makefile generator, the current CUDA support in CMake is not straight-forward at all. It would be nice if there was a way to merge the straight-forward "native" solution for the Unix makefile generator, and keep the already available wrapper scripts and custom commands (as needed) for non-Make generators.
(0025548)
Bill Hoffman (manager)
2011-02-22 15:39

 So, a while back, I added a language to custom command thing for the IDE generators. It was for the java support (which is broken right now...). Anyway, that stuff might work. James you might want to test the patch and see if it does work on a VS IDE build. If would be really good if we could get some nightly testing on mac, windows, and linux for this cuda stuff so we can make sure it actually works.
(0025550)
Peter Colberg (reporter)
2011-02-22 16:05

 The patch as is will probably not work on a VS IDE build, it is still missing a platform module (see Linux-NVCC-CUDA.cmake). I did not write such a module yet, as I have no means to test it.

James, for testing, should I prepare a Windows platform module, or could you just use the Linux module as a template and substitute the correct flags for NVCC?

On a side note, the patch entirely removes the upstream FindCUDA.cmake. This is of course not intended to be merged into upstream as is, however I do think that the FindCUDA module should focus on finding all the libraries and include directories, and not the compiler.
The overall goal of this patch is to make CUDA a first class language in CMake, to be used just like C, C++, Fortan, Java, ... by adding CUDA to the project() languages or calling enable_language(CUDA).
(0025551)
James Bigler (developer)
2011-02-22 17:11

 So Bill, there are facilities for adding custom commands to CMake languages? Do I need to poke around the Java code for this?

The custom command would be nice when invoking nvcc, but how would it handle changing the source file level dependencies in the IDE?

One of my biggest concerns with this going forward is maintaining two separate code paths for backward compatibility. True we could say that enable_language(CUDA) behaves differently, but if someone uses enable_language(CUDA) with the existing FindCUDA machinery, strange things could possibly happen to the .cu files added to the add_library and add_executable commands that are part of cuda_add_library and cuda_add_library.
(0025825)
Peter Colberg (reporter)
2011-03-18 18:26

 I updated the “native” CUDA patch to CMake 2.8.4, along with a few changes:

Search CMake CUDA compiler in CUDA_TOOLKIT_ROOT_DIR environment variable.

Search more paths, as found in FindLua51.cmake.
    
Use the environment variable CUDA_TOOLKIT_ROOT_DIR to be compatible with the upstream FindCUDA.cmake, and add the environment variables using HINTS. Use PATH_SUFFIXES in find_path and find_library.
(0025826)
Peter Colberg (reporter)
2011-03-18 18:30

 The recommended usage with CUDA as a required language would be:

project(foo C CXX CUDA)
find_package(CUDA QUIET REQUIRED)

The recommended usage with CUDA as an optional language would be:

project(foo)
find_package(CUDA QUIET)
if(CUDA_FOUND)
  enable_language(CUDA)
endif()
(0025835)
James Bigler (developer)
2011-03-21 00:45

 I'm still questioning how this would work in a backward compatible way.

One such backward compatibility issue is:
How are PTX targets supported?
(0025839)
Peter Colberg (reporter)
2011-03-21 10:25

 PTX targets are implemented analogous to C/C++ assembly code targets, so to compile a CUDA source file to PTX you invoke "make <source>.s".
See the variable CMAKE_CUDA_CREATE_ASSEMBLY_SOURCE (in Modules/Compiler/NVCC-CUDA.cmake), which defines the parameters passed to CMAKE_CUDA_COMPILER for PTX creation. This further adds rules to create preprocessed CUDA source using "make <source>.i".
(0025841)
James Bigler (developer)
2011-03-21 11:30

 How would this work if I used the PTX output as an input to another build rule?

In my build system I generated PTX in one rule, then I take this PTX and inline it all into a CPP file using CUDA's bin2c executable. This CPP file then gets compiled into my library.
(0025842)
Peter Colberg (reporter)
2011-03-21 12:12

 If the CMake core supported compiling a C/C++ source file to assembly code and the assembly code to an object file, then this would be supported for CUDA PTX assembly as well.

Based on your work flow, I assume you use the CUDA driver instead of the CUDA runtime library? If I am not mistaken, CUDA 4.0 will add support for reading fat binary objects with the CUDA driver API, which would (hopefully) make manual inlining with bin2c unneeded.
(0025843)
James Bigler (developer)
2011-03-21 12:31

 Yeah, I don't know the extent of CMake support for dealing with assembly code.

I'm not sure your 'make <source>.s' makefile stuff will work at all for Visual Studio projects. I need explicit rules for VS.

As far as CUDA 4.0's ability to support reading fat binaries, that simply won't work for us. Our tool reads the PTX directly and does our own transformations on the PTX before emitting a new PTX kernel to the driver. My app needs to be able to read the PTX strings before the driver ever sees it.
(0025845)
Peter Colberg (reporter)
2011-03-21 15:01

 That is an interesting workflow indeed. Off topic, may I ask in which way you transform the PTX code?

So apparently there is support in CMake for compiling assembler sources:

http://www.vtk.org/Wiki/CMake/Assembler [^]

Maybe one could add an ASM_CUDA_PTX language to compile PTX assembly?
(0025846)
Peter Colberg (reporter)
2011-03-21 15:05

 It looks feasible, I might try to implement this:

http://www.vtk.org/Wiki/CMake/Assembler#How_to_add_support_for_other_assembler_.22dialects.22 [^]
(0025848)
James Bigler (developer)
2011-03-21 15:43

 I think the Assembler stuff is designed to take assembler files and generate object code, so I'm not sure how you mean to use it. I need CUDA C -> PTX and CUDA C -> PTX -> bin2c -> ObjectCode.

> Off topic, may I ask in which way you transform the PTX code?

My project is the NVIDIA OptiX SDK. We take input PTX from users with special ray tracing extensions and generate a complete working program. One of the transformations on the PTX we employ is to rewrite the stub functions in user PTX into their respective implementations. Users of OptiX were able to utilize recursive ray tracing long before recursive function calls were supported by CUDA or the hardware.
(0028616)
Peter Colberg (reporter)
2012-02-17 11:44

 This wishlist bug has been open quite a while, so I would like to know if there is anything I can do to ease its implementation. I already implemented the Linux part of CUDA language support using native CMake commands, but do you see a way to implement whatever is missing on the Windows side to make this happen?

I have no problem maintaining CMake-CUDA myself, and advise users of my projects to use it, but it would be very nice to see this integrated into all GNU/Linux distributions in the near future to everyone's benefit.

Thanks,
Peter
(0029783)
Peter Colberg (reporter)
2012-06-20 16:11

 Hi,

I updated the patch with CUDA support for CMake to version 2.8.8.

The patch so far only supports GNU/Linux, but could easily be extended to platforms with makefile support.

With CMake 2.8.8, the version of the CUDA compiler is detected, and stored in CMAKE_CUDA_COMPILER_VERSION.


The CUDA package, which finds CUDA libraries, supports COMPONENTS, e.g.

find_package(CUDA QUIET REQUIRED COMPONENTS cuda cudart)


CUDA compiler flags may be preseeded using builtin CMake infrastructure, e.g.

# CMakeLists.txt

set(CMAKE_USER_MAKE_RULES_OVERRIDE "cmake/platform.cmake")
project(<name> CXX CUDA)

# cmake/platform.cmake

if(DEFINED CMAKE_CUDA_COMPILER_ID)
  if(CMAKE_CUDA_COMPILER_ID STREQUAL "NVCC")
      set(CMAKE_CUDA_FLAGS_INIT "-Xcompiler -fPIC -Xptxas -v -arch=compute_12 -code=compute_12,sm_13,sm_20")

  else()
    message(WARNING "Unsupported CUDA compiler: ${CMAKE_CUDA_COMPILER_ID}")
  endif()
endif()


The compiler and flags may alternatively be overridden via environment variables:

# use system GCC compiler (e.g. GCC 4.6), instead of gcc in PATH (e.g. GCC 4.7, unsupported by CUDA 4.2)
CUDACC="nvcc --compiler-bindir=/usr/bin" cmake ...

# keep intermediate files
CUDACCFLAGS=--keep cmake ...

Regards,
Peter
(0030447)
David Cole (manager)
2012-08-11 21:42

 Sending old, never assigned issues to the backlog.

(The age of the bug, plus the fact that it's never been assigned to anyone means that nobody is actively working on it...)

If an issue you care about is sent to the backlog when you feel it should have been addressed in a different manner, please bring it up on the CMake mailing list for discussion. Sign up for the mailing list here, if you're not already on it: http://www.cmake.org/mailman/listinfo/cmake [^]

It's easy to re-activate a bug here if you can find a CMake developer who has the bandwidth to take it on, and ferry a fix through to our 'next' branch for dashboard testing.
(0041805)
Kitware Robot (administrator)
2016-06-10 14:28

 Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

 Issue History
Date Modified Username Field Change
2011-02-22 11:32 Peter Colberg New Issue
2011-02-22 11:32 Peter Colberg File Added: cmake-cuda-2.8.3.patch
2011-02-22 11:44 Bill Hoffman Note Added: 0025532
2011-02-22 12:01 Peter Colberg Note Added: 0025533
2011-02-22 12:17 James Bigler Note Added: 0025534
2011-02-22 12:28 Peter Colberg Note Added: 0025535
2011-02-22 12:38 Peter Colberg Note Added: 0025536
2011-02-22 12:48 James Bigler Note Added: 0025537
2011-02-22 13:00 Peter Colberg Note Added: 0025538
2011-02-22 13:11 Peter Colberg Note Added: 0025539
2011-02-22 13:12 James Bigler Note Added: 0025540
2011-02-22 13:13 Peter Colberg Note Added: 0025541
2011-02-22 13:14 James Bigler Note Added: 0025542
2011-02-22 13:14 Peter Colberg Note Added: 0025543
2011-02-22 13:20 James Bigler Note Added: 0025544
2011-02-22 13:20 James Bigler Note Edited: 0025544
2011-02-22 13:57 Peter Colberg Note Added: 0025545
2011-02-22 15:39 Bill Hoffman Note Added: 0025548
2011-02-22 16:05 Peter Colberg Note Added: 0025550
2011-02-22 17:11 James Bigler Note Added: 0025551
2011-03-18 18:23 Peter Colberg File Added: cmake-cuda-2.8.4.patch
2011-03-18 18:26 Peter Colberg Note Added: 0025825
2011-03-18 18:30 Peter Colberg Note Added: 0025826
2011-03-21 00:45 James Bigler Note Added: 0025835
2011-03-21 10:25 Peter Colberg Note Added: 0025839
2011-03-21 11:30 James Bigler Note Added: 0025841
2011-03-21 12:12 Peter Colberg Note Added: 0025842
2011-03-21 12:31 James Bigler Note Added: 0025843
2011-03-21 15:01 Peter Colberg Note Added: 0025845
2011-03-21 15:05 Peter Colberg Note Added: 0025846
2011-03-21 15:43 James Bigler Note Added: 0025848
2011-03-22 12:41 Peter Colberg File Added: CMake-CUDA-2.8.4-0-g7f92dd3.patch
2011-07-28 22:35 Peter Colberg File Added: cmake-cuda-2.8.5-0-gda84f60.patch
2012-01-11 11:34 Peter Colberg File Added: cmake-cuda-2.8.7-0-ga9ce0fa.patch
2012-02-17 11:44 Peter Colberg Note Added: 0028616
2012-02-17 11:55 Peter Colberg Note Added: 0028617
2012-02-17 11:57 Peter Colberg Note Deleted: 0028617
2012-06-20 15:56 Peter Colberg File Added: cmake-cuda-2.8.8-2-ga06ff77.patch
2012-06-20 16:11 Peter Colberg Note Added: 0029783
2012-08-11 21:42 David Cole Status new => backlog
2012-08-11 21:42 David Cole Note Added: 0030447
2016-06-10 14:28 Kitware Robot Note Added: 0041805
2016-06-10 14:28 Kitware Robot Status backlog => resolved
2016-06-10 14:28 Kitware Robot Resolution open => moved
2016-06-10 14:28 Kitware Robot Assigned To => Kitware Robot
2016-06-10 14:31 Kitware Robot Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team