[CMake] Bad documentation of the LINK_INTERFACE_LIBRARIES property and other transitive linking topics

[Michael Hertling]

On 11/23/2011 10:25 AM, Alan W. Irwin wrote:
> cmake-2.8.6 has the following documentation of the
> LINK_INTERFACE_LIBRARIES property for targets:
> 
>    LINK_INTERFACE_LIBRARIES
>         List public interface libraries for a shared library or executable.
> 
>         By default linking to a shared library target transitively links to
>         targets with which the library itself was linked.  For an executable
>         with exports (see the ENABLE_EXPORTS property) no default transitive
>         link dependencies are used.  This property replaces the default
>         transitive link dependencies with an explicit list.  When the target
>         is linked into another target the libraries listed (and recursively
>         their link interface libraries) will be provided to the other target
>         also.  If the list is empty then no transitive link dependencies will
>         be incorporated when this target is linked into another target even if
>         the default set is non-empty.  This property is ignored for STATIC
>         libraries.
> 
> I believe the reference to executables above is just plain
> wrong/misleading.  For example, the FAQ states that
> LINK_INTERFACE_LIBRARIES simply lists "the libraries that should be
> transitively included in the link by CMake" with the clear implication
> (also confirmed by experiment below) that this target property affects
> _everything that links to the target whose property is being set_.
> But nothing links to an executable so why are executables mentioned at
> all?

Michael has already mentioned the ENABLE_EXPORTS property; see the
following project for a typical example regarding plugins on *nix:

CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
PROJECT(EXPORTS C)
SET(CMAKE_VERBOSE_MAKEFILE ON)
SET(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "")
FILE(WRITE ${CMAKE_BINARY_DIR}/main.c
"#include <dlfcn.h>
#include <stdio.h>
void internal(void){printf(\"internal()\\n\");}
int main(void)
{
    void *p=dlopen(\"./libplugin.so\",RTLD_NOW);
    void (*f)(void)=dlsym(p,\"external\");
    f();
}\n")
ADD_EXECUTABLE(main main.c)
TARGET_LINK_LIBRARIES(main dl)
SET_TARGET_PROPERTIES(main PROPERTIES
    ENABLE_EXPORTS TRUE)
FILE(WRITE ${CMAKE_BINARY_DIR}/plugin.c
"void external(void){internal();}\n")
ADD_LIBRARY(plugin MODULE plugin.c)
TARGET_LINK_LIBRARIES(plugin main)

The plugin provides the function external() that calls the function
internal() which is in turn defined in the main executable. If the
latter is linked without the -Wl,--export-dynamic switch - enabled
by ENABLE_EXPORTS - it simply won't run because internal()'s symbol
can not be found by the plugin. However, on ELF platforms, linking
against executables doesn't require to change the link command, but
the TARGET_LINK_LIBRARIES() command constrains the activation of the
ENABLE_EXPORTS property on the executable, so you can't accidentally
forget to export symbols from an executable when a plugin needs them.
Disable TARGET_LINK_LIBRARIES(plugin ...) and SET_TARGET_PROPERTIES()
to see it fail. Thus, one can very well link against an executable -
possibly only, say, formally - so it's not pointless to mention an
executable in a target's LINK_INTERFACE_LIBRARIES property.

Note the following remarks:

(1) For GCC on Linux, CMake adds -rdynamic a.k.a. -Wl,--export-dynamic
to an executable's link command line automatically, cf. the comment in
Modules/Platform/Linux-GNU.cmake, so CMAKE_SHARED_LIBRARY_LINK_C_FLAGS
must be emptied in the above-noted example to see the related effects.

(2) For the ENABLE_EXPORTS property, CMake's documentation states: "On
all platforms a target-level dependency on the executable is created
for targets that link to it." AFAICS, this is not true for ELF plat-
forms: If one touches main.c and rebuilds, only the main target is
rebuilt but not the plugin. That's no harm because rebuilding the
executable wouldn't have any effect on the plugin, but perhaps,
the documentation should be fixed in this regard.

> Here is the background for why I am looking into using this property:
> 
> I am told by Orion Poplawski (the guy who packages PLplot for Fedora)
> that "rpmlint" warns whenever Linux libraries are overlinked
> (presumably because that overlinking brings in unneeded rpm library
> dependencies).  I have confirmed for one PLplot library where he gave
> the detailed warning messages, that "ldd -u" lists exactly the same
> unused direct dependencies as rpmlint. These rpmlint and "ldd -u"
> warnings are the direct result of the default transitive linking for
> shared libraries used in CMake.
> 
> So I have been investigating how to avoid transitive linking for
> shared libraries by consulting the CMake FAQ, cmake man page, and
> previous threads on this list that have "transitive" in the subject
> line. Apparently you are supposed to create an empty
> LINK_INTERFACE_LIBRARIES target property.  However, it is not clear
> anywhere which target(s) in a chain of direct dependencies should have
> an empty LINK_INTERFACE_LIBRARIES property.
> 
> Here is a test CMakeList.txt file (adapted from a previous thread on
> transitive linking) to investigate this question.
> 
> CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
> PROJECT(TRANSLINK C)
> 
> # Test results for main executable
> # which directly depends on just libE
> # which directly depends just on libD 
> # which directly depends just on libC
> # which directly depends just on libB
> # which directly depends just on libA
> # All libraries are shared.
> 
> FILE(WRITE ${CMAKE_BINARY_DIR}/a.c "#include <stdio.h>\nvoid a(void){printf(\"hello, world\\n\");}\n")
> ADD_LIBRARY(A SHARED a.c)
> #TARGET_LINK_LIBRARIES(A LINK_INTERFACE_LIBRARIES)
> 
> FILE(WRITE ${CMAKE_BINARY_DIR}/b.c "void b(void){a();}\n")
> ADD_LIBRARY(B SHARED b.c)
> TARGET_LINK_LIBRARIES(B A)
> #TARGET_LINK_LIBRARIES(B LINK_INTERFACE_LIBRARIES)
> 
> FILE(WRITE ${CMAKE_BINARY_DIR}/c.c "void c(void){b();}\n")
> ADD_LIBRARY(C SHARED c.c)
> TARGET_LINK_LIBRARIES(C B)
> #TARGET_LINK_LIBRARIES(C LINK_INTERFACE_LIBRARIES)
> 
> FILE(WRITE ${CMAKE_BINARY_DIR}/d.c "void d(void){c();}\n")
> ADD_LIBRARY(D SHARED d.c)
> TARGET_LINK_LIBRARIES(D C)
> TARGET_LINK_LIBRARIES(D LINK_INTERFACE_LIBRARIES)
> 
> FILE(WRITE ${CMAKE_BINARY_DIR}/e.c "void e(void){d();}\n")
> ADD_LIBRARY(E SHARED e.c)
> TARGET_LINK_LIBRARIES(E D)
> #TARGET_LINK_LIBRARIES(E LINK_INTERFACE_LIBRARIES)
> 
> FILE(WRITE ${CMAKE_BINARY_DIR}/main.c "int main(){e();}\n")
> ADD_EXECUTABLE(main main.c)
> TARGET_LINK_LIBRARIES(main E)
> #TARGET_LINK_LIBRARIES(main LINK_INTERFACE_LIBRARIES)
> 
> Note, the LINK_INTERFACE_LIBRARIES target property is emptied for the D
> library with the
> 
> TARGET_LINK_LIBRARIES(D LINK_INTERFACE_LIBRARIES)
> 
> command, but all other such commands are commented out.  The
> documentation of TARGET_LINK_LIBRARIES(... LINK_INTERFACE_LIBRARIES)
> implies equivalance of the above with
> 
> SET_TARGET_PROPERTIES(D PROPERTIES LINK_INTERFACE_LIBRARIES "")
> 
> and I have also confirmed that experimentally.
> 
> In sum, from this experiment it looks like I will have to set
> LINK_INTERFACE_LIBRARIES to empty for all PLplot library targets that
> depend on other library targets (e.g., B, C, D, and E, above, but _not_
> main or A) created by our build system to comprehensively turn off
> transitive linking and avoid overlinking that rpmlint is complaining
> about.

AFAIK, yes. Provided that you establish all *immediate* dependencies
correctly via TARGET_LINK_LIBRARIES(), you might drop the *mediate*
ones - pulled into the link command line by CMake - by explicitly
unsetting the LINK_INTERFACE_LIBRARIES properties. This cares for
a minimal set of DT_NEEDED tags across all dynamic binaries created
by your project, but see below for a possibly important implication.

> However, I am concerned about the safety of doing this.  For example,
> a decade or so ago Linux had a buggy run-time loader that would only
> work if you used transitive linking so that became the autotools
> practice (at least at that time, it may have changed since I quit
> using autotools) and I assume that is why transitive linking is the
> default with CMake. Are there still enterprise edition Linux distros
> out there where turning off transitive linking would cause problems?
> Does non-transitive linking cause problems on former or current
> Windows or Mac OS X platforms?

You are looking at this concern solely from the perspective of the
runtime linker, but the binutils' ld can also use DT_NEEDED tags to
find shared libraries for symbol resolution. If a mediately needed
shared library is not mentioned in the link command line explictly,
ld steps from tag to tag in order to find the required library, and
the locations it looks into during this search can be influenced in
various ways, see ld's manpage w.r.t. the -rpath-link switch. There-
with, you are effectively taking the same risks as with using LINK_
DIRECTORIES() that the linker will possibly be lured into the wrong
directories - subtle breakages included. Regarding project-internal
library targets with empty LINK_INTERFACE_LIBRARIES, CMake replaces
a prerequisite library's full path with a -Wl,-rpath-link switch to
provide maximal information to ld where the concerned library is to
be searched, but see [1] for an example failing nevertheless. With
mediately needed external libraries, there's no -rpath-link, and
accordingly, the risks are getting higher.

> Because of these concerns I will probably enable transitive linking by
> default but have an option (which defaults to OFF) to disable
> transitive linking with an associated docstring that warns that ON is
> experimental and may not work on all platforms.

Instead of doing so, you might choose a configuration for packaging,
e.g. RELEASE or even a custom one "PACKAGE" or the like, and cut the
targets' transitive link dependencies for that configuration only by
emptying the LINK_INTERFACE_LIBRARIES_PACKAGE properties throughout
your project. In this way, you could enjoy CMake's safety-conscious
default behavior during development, and minimize the usage of DT_
NEEDED tags when it comes to packaging.

> All this to quiet rpmlint warnings....

Out of curiosity - I have not worked with RPM for ages: Are these
warnings and the related overlinking due to transitive dependencies
really an issue or just an inconvenience? Personally, I distinguish
between real overlinking, i.e. pulling in libraries not used at all,
and formal overlinking by DT_NEEDED tags for mediate prerequisites,
or in other words: Real overlinking means the dependency graph has
unnecessary nodes, and formal overlinking means is has unnecessary
edges. Of course, the former is a real penalty, but is the latter
also bad? If an executable X is linked against a shared library B,
and B against a shared A, is it really critical if A explicitly
appears among the dependencies of X, as A *is* needed for X?
Perhaps, you can share some experiences from your practice.

Regards,

Michael

[1] http://www.mail-archive.com/cmake@cmake.org/msg39426.html

PS: What would be quite welcome w.r.t. this entire issue is a
possibility to mention a shared library in the linker command
line and have ld use it for symbol resolution only, but not
write a DT_NEEDED tag to the final binary.