[CMake] Building a repo with multiple applications and install process

[David Jobet]

Hello,

so I didn't get a lot of answers, here's a quick POC which only uses
add_subdirectory() / include_guard() for more questions.

/CMakeLists.txt
/app1/CMakeLists.txt
/app2/CMakeLists.txt
/common/CMakeLists.txt

We have app1 depending on common, and app2 depending on common.
All 3 projects are living in the same mono-repo.

With those CMakeLists.txt, I'm able to build/install app1, app2, and
common as standalone projects with the following command line
e.g for app1 :
mkdir build_app1 && cd build_app1 && cmake -DCMAKE_INSTALL_PREFIX=
/path/to/app1 && make && DESTDIR=dist make install

I'm also able to build all apps at once using the following
mkdir build_all && cd build_all && cmake -DCMAKE_INSTALL_PREFIX=
/path/to/root && make && DESTDIR=dist make install

See below for the content of the files.

Now the question :
Do you think it would be possible to use the "build all" approach to
populate all .o, libs and executables, then to reconfigure the build
dir, say for app1 so I can then issue "make install" only for app1 ?
This would involve the "build all" project to have the same layout as
the "app1" project which currently is not the case.
In the "build all" project, app1 is a directory
In the "app1" project, app1 is a binary

Thanks for your help

David

---------------------------------------------------
/CMakeLists.txt
cmake_minimum_required(VERSION 3.10 FATAL_ERROR)

project(main-klib CXX)

add_subdirectory(app1)
add_subdirectory(app2)
add_subdirectory(common)

---------------------------------------------------
/app1/CMakeLists.txt
cmake_minimum_required(VERSION 3.10 FATAL_ERROR)
include_guard(GLOBAL)

project(app1 CXX)

add_subdirectory(../common common)

add_executable(app1 main.cc)
target_link_libraries(app1 common::lib)

install(
   TARGETS app1
   RUNTIME DESTINATION bin
)

---------------------------------------------------
/app2/CMakeLists.txt
cmake_minimum_required(VERSION 3.10 FATAL_ERROR)
include_guard(GLOBAL)

project(app2 CXX)

add_subdirectory(../common common)

add_executable(app2 main.cc)
target_link_libraries(app2 common::lib)

install(
   TARGETS app2
   RUNTIME DESTINATION bin
)

---------------------------------------------------
/common/CMakeLists.txt
cmake_minimum_required(VERSION 3.10 FATAL_ERROR)
include_guard(GLOBAL)

project(common CXX)

add_library(common common.cc)
add_library(common::lib ALIAS common)
target_include_directories(common PUBLIC ${PROJECT_SOURCE_DIR})

install(
   FILES common.py
   DESTINATION site-packages
)

On Tue, Feb 5, 2019 at 10:05 AM David Jobet <djobet at tower-research.com> wrote:
>
> Hello,
>
> at work, we have a mono-repo with multiple applications/libs (dozens).
> The build phase is ok, but I'm not sure about the release process.
>
> When we release, we release one application at a time.
> (CMAKE_SKIP_INSTALL_ALL_DEPENDENCY is true)
> In order to speed up releases, we always perform an incremental build.
>
> Unfortunately, we don't have one unique release process :
> process 1 :
> - a Jenkins pipeline executes some automatic tests then release the
> binary to production. This Jenkins pipeline only builds this single
> application, then executes the install step, then packages the binary
> with some auxiliary files for deployment in prod.
> process 2 :
> - the whole source tree is built regularly through Jenkins, then, from
> another Jenkins pipeline, an install step will be performed in the
> last built directory to deploy only the required application
>
> Both process 1 and process 2 are built in our CMakeLists.txt.
>
> Process 1 just uses regular install directives + ninja install
> Pros : simple
> Cons : install step can be costly
>
> Install step can be costly because, as the build directory is not
> emptied, the install step will install every single binaries left over
> from a previous build that have an install rule.
> Also, we have install directives for non binary files (python files
> for example) which will be installed unconditionally every time.
>
> Process 2 is not triggered through the install step but as a regular
> build target. Under the hood, the build step will add a POST_BUILD
> step attached to the target that will invoke "cmake -P
> ${CMAKE_BINARY_DIR}/cmake_install.cmake -DCOMPONENT=${component}"
> Pros : more "chirurgical", only install what's required
> Cons : - if an application depends on several components, we need to
> describe this in cmake (dependencies are described twice : once for
> the build, once for the install)
>        - need to maintain an extra "non standard" layer (albeit a small layer)
>
> At this point, I'd like to ask if you see simple steps we could take
> to stay as simple and standard as possible without paying the cons
> (lenghty install step, double description of dependencies, extra layer
> to maintain).
>
> I have a proposal of my own, I'm just not sure this is technically
> feasible. I will definitively run a POC sometime, but I thought I
> would run it by you first to get your advice/experience.
>
> So maybe the problem is we have one monolithic CMake system where all
> apps are described.
> What if every single application had its own independent CMake system
> (while still being able to depend on common libs, that needs to be
> built only once) ?
> One app would describe its dependencies, the install step would then
> stay simple, and only the reachable install directives would be
> triggered in a per-app build.
>
> Is it something which is feasible ? How would you implement it ? (I
> thought about ExternalProject_Add but have no experience with it and
> I'm not sure it would work in that case ?)
>
> Also, we would still need to be able to build all applications in one
> single command.
> Do you think creating a "meta" cmake, equivalent to what we have right
> now, but on top of those independent, per-app cmake, is feasible ?
> (Again, I guess using ExternalProject_Add) ?
>
> Thanks very much for your help
>
> David