[CMake] One build, multiple compilers and packages

[Craig Scott]

Hi. First, apologies for the length. If you are not interested in mixing
different compilers and generating multiple packages all within one build,
you can probably stop reading now.

After trying various approaches and not being entirely satisfied with any,
I thought I'd seek the collective wisdom on this list. The details below
define the scenario, the main problems faced and some things that have been
tried to address them. Comments, ideas or suggestions would be most welcome.

SCENARIO:

We have a large code base for which different components need to be built
with different compilers. Some parts are desktop software (applications,
libraries) and others are firmware for embedded devices. All are pulled
together into a small number of packages using CPack. Some of those
packages share common items and there is no correlation between compilers
and packages. Most items are included in more than one of the packages. For
instance, firmware might be included in all packages, a particular desktop
application or library in a subset of the packages, etc. On a given
platform, some packages will have one format and other packages will have a
different format (eg a mix of RPM and tarball). We create packages for at
least Windows and Linux.

Most of the source code is in a single git repository and after testing
different strategies, this is an arrangement we want to keep. There are a
couple of components being built using CMake's ExternalProject feature,
either because they use a different build system that is currently too
complex for us to change to CMake or the source code comes from a 3rd party.


PROBLEMS:

   1. CMake assumes you only want to use one compiler for an entire build
   tree.
   2. CPack assumes you are only building one package. Some package
   generators allow you to do a component-based setup where multiple packages
   can be generated, but this is not supported for all package generators (eg
   the NSIS package generator won't produce multiple packages).


APPROACHES TRIED (feedback welcome):

The first of the above two issues is the more problematic. I've spent
considerable time experimenting with ExternalProject as a potential way of
addressing various issues, including the "one compiler per build tree"
restriction. What I found was that dependency handling and integration with
CPack becomes much more complicated, and in some cases has deficiencies not
present in a single unified CMake build. For example, once you have
successfully built an external project, the top level build won't rebuild
it again unless something about its configuration changes. Thus, if you
subsequently make changes to a source file, simply building the top level
project won't work because it still thinks the sub project is up to date.
This is because ExternalProject uses time stamps rather than forwarding the
build request to the sub project to let it decide if anything needs to be
built. This kinda makes sense, since an ExternalProject is meant to
encompass everything from download through to install, but this is not how
we are trying (and needing) to use it. ExternalProject starts to become
less useful when applied to source code you are actively working on.

Effectively, what we really need is the ability to let add_subdirectory()
create a whole new build with a potentially different compiler, but still
have it aware of all the other targets and dependencies and also to
propagate all its own the targets, dependencies, etc. up to the parent. We
experimented with exporting targets from the ExternalProject's using the
export features of the install command, but this was messy and ultimately
still suffers from the "isn't aware of changes to the source" problem. It
also has implications for stripping of targets (for an example, see
http://public.kitware.com/Bug/view.php?id=14157 ). Packaging also gets much
more cumbersome when ExternalPackage is involved.

For the second problem, my current workaround is to create a custom target
which re-configures and re-builds the project for each package I need to
create. The targets only need to get built once, but the package details
change each time and a different package gets built. In essence, the build
calls itself with a set of different configurations, which seemed a bit
mind-bending at first, but was fairly simple to implement and didn't seem
to have much in the way of negative side effects. The approach works
reliably, but it feels like a roundabout solution to something that
CMake/CPack should be able to handle much more cleanly. In a similar vein
to the suggestion above, it would be really nice if add_subdirectory() had
the ability to define a package that was not affected or did not affect
other directories. Thus, you could have one package per sub directory. Any
common variables could be defined at the parent level and would propagate
down to the sub directories. Alas, this is not something that
add_subdirectory() currently supports.

I'd be interested in hearing what other people have done or would do to
address the above problems for the specified scenario. I'm happy to discuss
in more detail things I've tried if people are interested, but hopefully
the above will stimulate some ideas or discussion about how these two
problems can be better handled with CMake/CPack. These are great tools for
which I'm sure these limitations can be overcome, either through methods I
haven't considered or improvements to the tools themselves.

-- 
Craig Scott
Melbourne, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20130826/c714c442/attachment-0001.htm>