[CMake] ExternalProjects_Add limitations

[Rob McDonald]

All,

Thanks for all your work on CMake.  I really hate toolchain problems and
CMake makes them almost bearable.  The entire community of developers
trying to make cross platform programs would be far worse off without CMake.

My project is a big open source C++ project with lots of library
dependencies (OpenGL, GLM, GLEW, CppTest, FLTK, LibXml2, LibJPEG, soon
adding Eigen3, STEPCODE).

In the past, we've left all the dependencies up to the developer.  This
limits us to truly dedicated (compensated) high-level wizards.  Even then,
you either spend two days working out the dependencies or some other wizard
sends you a 200MB zip file of precompiled libraries.  We believe this is a
significant problem towards attracting developers.  It also precludes us
from having any control/knowledge of what versions of libraries the
developers are actually using.  Not the best things for trying to build an
open source community.

I am determined to wrestle this beast.  Here is my plan...

We will embed the dependencies in our source.  This may be through git
submodules, direct includsion, CMake ExternalProject_Add, or a combination
thereof.

The embedded dependencies should be optional where appropriate -- for
platforms with extensive package libraries, we want to enable our program
to search for and use pre-installed libraries.

For everyone else, I believe that relying on the embedded dependencies will
be the primary mode for development.  So, we want the build environment to
'look normal' for this case.  I.e. project source files should show up in
IDEs, various make targets should be exposed.

We would like to use the library's own build files.  Whether CMake,
./configure, or something else, we presume that the project developers know
what they're doing and we don't want to second-guess them or recreate the
wheel.

We would also like to use the library's own FindXXX.cmake routines.  Some
of these are more complex than just setting one path and one library.  We'd
like to avoid recreating all of that knowledge.  This has the added
advantage that we can make our core project's CMake setup nearly
independent of how the libraries are handled.


I have something mostly working based on ExternalProject_Add.  Following
the advice online, I've used it for _everything_, all the big libraries and
our main project.

I've worked around the problem of the top level project not tracking
file-level changes of the nested projects using a 'ForceBuild' macro for
just our main project.

However, this approach does not make the development environment 'look
normal' to Visual Studio.  None of the project source appears, none of the
build targets appear (doc, package install, test, etc).  Is there any way
to make the 'Everything is an ExternalProject' approach 'look normal' to
Visual Studio and other IDE's?

I have since tried moving to a two-class system, where my main project is
not ExternalProject_Add'ed, but the big dependencies are.  This of course
runs into the problem where the libraries are not built when the
FindXXX.cmake modules go looking for them, the configure phase fails and
nothing ever gets built.

I believe the solution to this is prolific use of ADD_DEPENDENCIES, but I
also believe that solution requires setting XXX_INCLUDE_DIR and
XXX_LIBRARIES manually rather than through FindXXX.cmake.  That seems
fragile and difficult to me.


I'd like a way to force the main project to build the ExternalProjects
before even continuing with the CMake.  Perhaps a conditional...

ExternalProject_Add( libA .... )
ExternalProject_Add( libB .... )
ExternalProject_Add( libC .... )


IF ( ExternalProjectsOutOfDate )
  ExternalProject_Add( libcheck
     DEPENDS libA libB libC
     URL ""
     UPDATE_COMMAND ""
     CONFIGURE_COMMAND ""
     BUILD_COMMAND ""
     INSTALL_COMMAND ""
   )
   # Touch CMakeLists.txt to force re-run of CMake on next build.
ELSE ( ExternalProjectsOutOfDate )

# Continue on merry way for 'normal' project.
# ...

ENDIF ( ExternalProjectsOutOfDate )
# EOF

However, I don't know how to make that conditional work, or to re-trigger
CMake as required.


I'm currently leaning towards an even more two-class decoupled system.  In
this approach, there would be two 'root' CMakeLists.txt in our project.
 One would be just for the large dependencies, the other would be for the
main project.  There would be no actual link between them.  The developer
would have to first build the dependencies, then build the main project.

Does anyone know of a project that uses this approach?  It would be nice to
have some level of communication so the main project could 'know' that the
libraries were out of date (newer version had been added to the project)
and tell the developer to go rebuild them.

Any help is appreciated,

Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20140105/94409da2/attachment.htm>