[CMake] Transitive library dependencies with parallel builds

[Michael Hertling]

On 02/29/2012 05:35 PM, Number Cruncher wrote:
> Do transitive dependencies reduce number of jobs that can be compiled in 
> parallel?
> 
> If I have two libraries A and B, with an executable C, whose 
> dependencies are described by:
> 
>    add_library(A ${A_SRC})
> 
>    add_library(B ${B_SRC})
>    target_link_libraries(B A)
> 
>    add_executable(C ${C_SRC})
>    target_link_libraries(C B)
> 
> I understand that when *linking* C, the transitive dependency A will be 
> added. However, if I build C in parallel "make -j N", will CMake build 
> libraries A and B simultaneously, or fully compile and link A before 
> starting compilation of B? I.e. just because the link steps are serial 
> dependencies, are the compilation steps? Would it be faster to do:
> 
>    add_library(A ${A_SRC})
> 
>    add_library(B ${B_SRC})
> 
>    add_executable(C ${C_SRC})
>    target_link_libraries(C B A)
> 
> Thanks.

Look at the following exemplary project:

CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
PROJECT(P C)
FILE(WRITE ${CMAKE_BINARY_DIR}/a.c "void a(void){}\n")
FILE(WRITE ${CMAKE_BINARY_DIR}/b.c "void b(void){a();}\n")
FILE(WRITE ${CMAKE_BINARY_DIR}/c.c "int main(void){b(); return 0;}\n")
ADD_LIBRARY(A SHARED a.c)
ADD_LIBRARY(B SHARED b.c)
ADD_EXECUTABLE(C c.c)
IF(TRANSITIVE)
    TARGET_LINK_LIBRARIES(B A)
    TARGET_LINK_LIBRARIES(C B)
ELSE()
    TARGET_LINK_LIBRARIES(C B A)
ENDIF()

Configure with -DTRANSITIVE=ON and inspect CMakeFiles/Makefile2:

CMakeFiles/A.dir/all:
CMakeFiles/B.dir/all: CMakeFiles/A.dir/all
CMakeFiles/C.dir/all: CMakeFiles/B.dir/all

With -DTRANSITIVE=OFF, these lines read:

CMakeFiles/A.dir/all:
CMakeFiles/B.dir/all:
CMakeFiles/C.dir/all: CMakeFiles/A.dir/all
CMakeFiles/C.dir/all: CMakeFiles/B.dir/all

The CMakeFiles/<X>.dir/all targets do:

$(MAKE) -f CMakeFiles/<X>.dir/build.make CMakeFiles/<X>.dir/build

Finally, CMakeFiles/<X>.dir/build in CMakeFiles/<X>.dir/build.make
does build target <X> completely, i.e. including the linking step.

Thus, the two-part transitive linking with -DTRANSITIVE=ON indeed
completes A before addressing B, so A and B can not be compiled in
parallel. In contrast, the one-part non-transitive linking with -D
TRANSITIVE=OFF allows for A and B to be compiled and even linked in
parallel since they haven't any interdependencies. So, with -j, the
latter is potentially faster than the former, but...

...reconsider what you're about to do: If B actually references A,
you might possibly not want to drop the TARGET_LINK_LIBRARIES(B A)
command. Run "readelf -d libB.so" on both results for -DTRANSITIVE
and you will see the difference. If A and B were static libraries,
CMake would lose the awareness that B must pull in A in the linker
command line.

In short, the answers to your questions are: N/Y, Y and Y.

Regards,

Michael